{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s_qNSzzyaCbD"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "jmjh290raIky"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "J0Qjg6vuaHNt"
   },
   "source": [
    "# 基于注意力的神经机器翻译"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AOpGoE2T-YXS"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://www.tensorflow.google.cn/tutorials/text/nmt_with_attention\">\n",
    "    <img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />\n",
    "    在 TensorFlow.org 上查看</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/zh-cn/tutorials/text/nmt_with_attention.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />\n",
    "    在 Google Colab 运行</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/zh-cn/tutorials/text/nmt_with_attention.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />\n",
    "    在 GitHub 上查看源代码</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/zh-cn/tutorials/text/nmt_with_attention.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />下载此 notebook</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8dEwzVWg0f-E"
   },
   "source": [
    "Note: 我们的 TensorFlow 社区翻译了这些文档。因为社区翻译是尽力而为， 所以无法保证它们是最准确的，并且反映了最新的\n",
    "[官方英文文档](https://www.tensorflow.org/?hl=en)。如果您有改进此翻译的建议， 请提交 pull request 到\n",
    "[tensorflow/docs](https://github.com/tensorflow/docs) GitHub 仓库。要志愿地撰写或者审核译文，请加入\n",
    "[docs-zh-cn@tensorflow.org Google Group](https://groups.google.com/a/tensorflow.org/forum/#!forum/docs-zh-cn)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CiwtNgENbx2g"
   },
   "source": [
    "此笔记本训练一个将西班牙语翻译为英语的序列到序列（sequence to sequence，简写为 seq2seq）模型。此例子难度较高，需要对序列到序列模型的知识有一定了解。\n",
    "\n",
    "训练完此笔记本中的模型后，你将能够输入一个西班牙语句子，例如 *\"¿todavia estan en casa?\"*，并返回其英语翻译 *\"are you still at home?\"*\n",
    "\n",
    "对于一个简单的例子来说，翻译质量令人满意。但是更有趣的可能是生成的注意力图：它显示在翻译过程中，输入句子的哪些部分受到了模型的注意。\n",
    "\n",
    "<img src=\"https://tensorflow.org/images/spanish-english.png\" alt=\"spanish-english attention plot\">\n",
    "\n",
    "请注意：运行这个例子用一个 P100 GPU 需要花大约 10 分钟。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tnxXKDjq3jEL"
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "try:\n",
    "    # %tensorflow_version 仅存在于 Colab\n",
    "    %tensorflow_version 2.x\n",
    "except Exception:\n",
    "    pass\n",
    "import tensorflow as tf\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import unicodedata\n",
    "import re\n",
    "import numpy as np\n",
    "import os\n",
    "import io\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wfodePkj3jEa"
   },
   "source": [
    "## 下载和准备数据集\n",
    "\n",
    "我们将使用 http://www.manythings.org/anki/ 提供的一个语言数据集。这个数据集包含如下格式的语言翻译对：\n",
    "\n",
    "```\n",
    "May I borrow this book?\t¿Puedo tomar prestado este libro?\n",
    "```\n",
    "\n",
    "这个数据集中有很多种语言可供选择。我们将使用英语 - 西班牙语数据集。为方便使用，我们在谷歌云上提供了此数据集的一份副本。但是你也可以自己下载副本。下载完数据集后，我们将采取下列步骤准备数据：\n",
    "\n",
    "1. 给每个句子添加一个 *开始* 和一个 *结束* 标记（token）。\n",
    "2. 删除特殊字符以清理句子。\n",
    "3. 创建一个单词索引和一个反向单词索引（即一个从单词映射至 id 的词典和一个从 id 映射至单词的词典）。\n",
    "4. 将每个句子填充（pad）到最大长度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kRVATYOgJs1b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from http://storage.googleapis.com/download.tensorflow.org/data/spa-eng.zip\n",
      "2646016/2638744 [==============================] - 3s 1us/step\n"
     ]
    }
   ],
   "source": [
    "# 下载文件\n",
    "path_to_zip = tf.keras.utils.get_file(\n",
    "    'spa-eng.zip', origin='http://storage.googleapis.com/download.tensorflow.org/data/spa-eng.zip',\n",
    "    extract=True)\n",
    "\n",
    "path_to_file = os.path.dirname(path_to_zip)+\"/spa-eng/spa.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rd0jw-eC3jEh"
   },
   "outputs": [],
   "source": [
    "# 将 unicode 文件转换为 ascii\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn')\n",
    "\n",
    "\n",
    "def preprocess_sentence(w):\n",
    "    w = unicode_to_ascii(w.lower().strip())\n",
    "\n",
    "    # 在单词与跟在其后的标点符号之间插入一个空格\n",
    "    # 例如： \"he is a boy.\" => \"he is a boy .\"\n",
    "    # 参考：https://stackoverflow.com/questions/3645931/python-padding-punctuation-with-white-spaces-keeping-punctuation\n",
    "    w = re.sub(r\"([?.!,¿])\", r\" \\1 \", w)\n",
    "    w = re.sub(r'[\" \"]+', \" \", w)\n",
    "\n",
    "    # 除了 (a-z, A-Z, \".\", \"?\", \"!\", \",\")，将所有字符替换为空格\n",
    "    w = re.sub(r\"[^a-zA-Z?.!,¿]+\", \" \", w)\n",
    "\n",
    "    w = w.rstrip().strip()\n",
    "\n",
    "    # 给句子加上开始和结束标记\n",
    "    # 以便模型知道何时开始和结束预测\n",
    "    w = '<start> ' + w + ' <end>'\n",
    "    return w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "opI2GzOt479E"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> may i borrow this book ? <end>\n",
      "b'<start> \\xc2\\xbf puedo tomar prestado este libro ? <end>'\n"
     ]
    }
   ],
   "source": [
    "en_sentence = u\"May I borrow this book?\"\n",
    "sp_sentence = u\"¿Puedo tomar prestado este libro?\"\n",
    "print(preprocess_sentence(en_sentence))\n",
    "print(preprocess_sentence(sp_sentence).encode('utf-8'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OHn4Dct23jEm"
   },
   "outputs": [],
   "source": [
    "# 1. 去除重音符号\n",
    "# 2. 清理句子\n",
    "# 3. 返回这样格式的单词对：[ENGLISH, SPANISH]\n",
    "def create_dataset(path, num_examples):\n",
    "    lines = io.open(path, encoding='UTF-8').read().strip().split('\\n')\n",
    "\n",
    "    word_pairs = [[preprocess_sentence(w) for w in l.split('\\t')]  for l in lines[:num_examples]]\n",
    "\n",
    "    return zip(*word_pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cTbSbBz55QtF"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> if you want to sound like a native speaker , you must be willing to practice saying the same sentence over and over in the same way that banjo players practice the same phrase over and over until they can play it correctly and at the desired tempo . <end>\n",
      "<start> si quieres sonar como un hablante nativo , debes estar dispuesto a practicar diciendo la misma frase una y otra vez de la misma manera en que un musico de banjo practica el mismo fraseo una y otra vez hasta que lo puedan tocar correctamente y en el tiempo esperado . <end>\n"
     ]
    }
   ],
   "source": [
    "en, sp = create_dataset(path_to_file, None)\n",
    "print(en[-1])\n",
    "print(sp[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OmMZQpdO60dt"
   },
   "outputs": [],
   "source": [
    "def max_length(tensor):\n",
    "    return max(len(t) for t in tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bIOn8RCNDJXG"
   },
   "outputs": [],
   "source": [
    "def tokenize(lang):\n",
    "    lang_tokenizer = tf.keras.preprocessing.text.Tokenizer(filters='')\n",
    "    lang_tokenizer.fit_on_texts(lang)\n",
    "\n",
    "    tensor = lang_tokenizer.texts_to_sequences(lang)\n",
    "\n",
    "    tensor = tf.keras.preprocessing.sequence.pad_sequences(tensor,\n",
    "                                                           padding='post')\n",
    "\n",
    "    return tensor, lang_tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eAY9k49G3jE_"
   },
   "outputs": [],
   "source": [
    "def load_dataset(path, num_examples=None):\n",
    "    # 创建清理过的输入输出对\n",
    "    targ_lang, inp_lang = create_dataset(path, num_examples)\n",
    "\n",
    "    input_tensor, inp_lang_tokenizer = tokenize(inp_lang)\n",
    "    target_tensor, targ_lang_tokenizer = tokenize(targ_lang)\n",
    "\n",
    "    return input_tensor, target_tensor, inp_lang_tokenizer, targ_lang_tokenizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GOi42V79Ydlr"
   },
   "source": [
    "### 限制数据集的大小以加快实验速度（可选）\n",
    "\n",
    "在超过 10 万个句子的完整数据集上训练需要很长时间。为了更快地训练，我们可以将数据集的大小限制为 3 万个句子（当然，翻译质量也会随着数据的减少而降低）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cnxC7q-j3jFD"
   },
   "outputs": [],
   "source": [
    "# 尝试实验不同大小的数据集\n",
    "num_examples = 30000\n",
    "input_tensor, target_tensor, inp_lang, targ_lang = load_dataset(path_to_file, num_examples)\n",
    "\n",
    "# 计算目标张量的最大长度 （max_length）\n",
    "max_length_targ, max_length_inp = max_length(target_tensor), max_length(input_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4QILQkOs3jFG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24000 24000 6000 6000\n"
     ]
    }
   ],
   "source": [
    "# 采用 80 - 20 的比例切分训练集和验证集\n",
    "input_tensor_train, input_tensor_val, target_tensor_train, target_tensor_val = train_test_split(input_tensor, target_tensor, test_size=0.2)\n",
    "\n",
    "# 显示长度\n",
    "print(len(input_tensor_train), len(target_tensor_train), len(input_tensor_val), len(target_tensor_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lJPmLZGMeD5q"
   },
   "outputs": [],
   "source": [
    "def convert(lang, tensor):\n",
    "    for t in tensor:\n",
    "        if t != 0:\n",
    "            print(\"%d ----> %s\" % (t, lang.index_word[t]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VXukARTDd7MT"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input Language; index to word mapping\n",
      "1 ----> <start>\n",
      "12 ----> me\n",
      "3428 ----> mude\n",
      "3 ----> .\n",
      "2 ----> <end>\n",
      "\n",
      "Target Language; index to word mapping\n",
      "1 ----> <start>\n",
      "4 ----> i\n",
      "602 ----> moved\n",
      "3 ----> .\n",
      "2 ----> <end>\n"
     ]
    }
   ],
   "source": [
    "print (\"Input Language; index to word mapping\")\n",
    "convert(inp_lang, input_tensor_train[0])\n",
    "print ()\n",
    "print (\"Target Language; index to word mapping\")\n",
    "convert(targ_lang, target_tensor_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rgCLkfv5uO3d"
   },
   "source": [
    "### 创建一个 tf.data 数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TqHsArVZ3jFS"
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = len(input_tensor_train)\n",
    "BATCH_SIZE = 64\n",
    "steps_per_epoch = len(input_tensor_train)//BATCH_SIZE\n",
    "embedding_dim = 256\n",
    "units = 1024\n",
    "vocab_inp_size = len(inp_lang.word_index)+1\n",
    "vocab_tar_size = len(targ_lang.word_index)+1\n",
    "\n",
    "dataset = tf.data.Dataset.from_tensor_slices((input_tensor_train, target_tensor_train)).shuffle(BUFFER_SIZE)\n",
    "dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qc6-NK1GtWQt"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([64, 16]), TensorShape([64, 11]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_input_batch, example_target_batch = next(iter(dataset))\n",
    "example_input_batch.shape, example_target_batch.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TNfHIF71ulLu"
   },
   "source": [
    "## 编写编码器 （encoder） 和解码器 （decoder） 模型\n",
    "\n",
    "实现一个基于注意力的编码器 - 解码器模型。关于这种模型，你可以阅读 TensorFlow 的 [神经机器翻译 (序列到序列) 教程](https://github.com/tensorflow/nmt)。本示例采用一组更新的 API。此笔记本实现了上述序列到序列教程中的 [注意力方程式](https://github.com/tensorflow/nmt#background-on-the-attention-mechanism)。下图显示了注意力机制为每个输入单词分配一个权重，然后解码器将这个权重用于预测句子中的下一个单词。下图和公式是 [Luong 的论文](https://arxiv.org/abs/1508.04025v5)中注意力机制的一个例子。\n",
    "\n",
    "<img src=\"./img/attention_mechanism.jpg\" width=\"500\" alt=\"attention mechanism\">\n",
    "\n",
    "输入经过编码器模型，编码器模型为我们提供形状为 *(批大小，最大长度，隐藏层大小)* 的编码器输出和形状为 *(批大小，隐藏层大小)* 的编码器隐藏层状态。\n",
    "\n",
    "下面是所实现的方程式：\n",
    "\n",
    "<img src=\"./img/attention_equation_0.jpg\" alt=\"attention equation 0\" width=\"800\">\n",
    "<img src=\"./img/attention_equation_1.jpg\" alt=\"attention equation 1\" width=\"800\">\n",
    "\n",
    "本教程的编码器采用 [Bahdanau 注意力](https://arxiv.org/pdf/1409.0473.pdf)。在用简化形式编写之前，让我们先决定符号：\n",
    "\n",
    "* FC = 完全连接（密集）层\n",
    "* EO = 编码器输出\n",
    "* H = 隐藏层状态\n",
    "* X = 解码器输入\n",
    "\n",
    "以及伪代码：\n",
    "\n",
    "* `score = FC(tanh(FC(EO) + FC(H)))`\n",
    "* `attention weights = softmax(score, axis = 1)`。 Softmax 默认被应用于最后一个轴，但是这里我们想将它应用于 *第一个轴*, 因为分数 （score） 的形状是 *(批大小，最大长度，隐藏层大小)*。最大长度 （`max_length`） 是我们的输入的长度。因为我们想为每个输入分配一个权重，所以 softmax 应该用在这个轴上。\n",
    "* `context vector = sum(attention weights * EO, axis = 1)`。选择第一个轴的原因同上。\n",
    "* `embedding output` = 解码器输入 X 通过一个嵌入层。\n",
    "* `merged vector = concat(embedding output, context vector)`\n",
    "* 此合并后的向量随后被传送到 GRU\n",
    "\n",
    "每个步骤中所有向量的形状已在代码的注释中阐明："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nZ2rI24i3jFg"
   },
   "outputs": [],
   "source": [
    "class Encoder(tf.keras.Model):\n",
    "    def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz):\n",
    "        super(Encoder, self).__init__()\n",
    "        self.batch_sz = batch_sz\n",
    "        self.enc_units = enc_units\n",
    "        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "        self.gru = tf.keras.layers.GRU(self.enc_units,\n",
    "                                       return_sequences=True,\n",
    "                                       return_state=True,\n",
    "                                       recurrent_initializer='glorot_uniform')\n",
    "\n",
    "    def call(self, x, hidden):\n",
    "        x = self.embedding(x)\n",
    "        output, state = self.gru(x, initial_state=hidden)\n",
    "        return output, state\n",
    "\n",
    "    def initialize_hidden_state(self):\n",
    "        return tf.zeros((self.batch_sz, self.enc_units))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "60gSVh05Jl6l"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Encoder output shape: (batch size, sequence length, units) (64, 16, 1024)\n",
      "Encoder Hidden state shape: (batch size, units) (64, 1024)\n"
     ]
    }
   ],
   "source": [
    "encoder = Encoder(vocab_inp_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "# 样本输入\n",
    "sample_hidden = encoder.initialize_hidden_state()\n",
    "sample_output, sample_hidden = encoder(example_input_batch, sample_hidden)\n",
    "print ('Encoder output shape: (batch size, sequence length, units) {}'.format(sample_output.shape))\n",
    "print ('Encoder Hidden state shape: (batch size, units) {}'.format(sample_hidden.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "umohpBN2OM94"
   },
   "outputs": [],
   "source": [
    "class BahdanauAttention(tf.keras.layers.Layer):\n",
    "    def __init__(self, units):\n",
    "        super(BahdanauAttention, self).__init__()\n",
    "        self.W1 = tf.keras.layers.Dense(units)\n",
    "        self.W2 = tf.keras.layers.Dense(units)\n",
    "        self.V = tf.keras.layers.Dense(1)\n",
    "\n",
    "    def call(self, query, values):\n",
    "        # 隐藏层的形状 == （批大小，隐藏层大小）\n",
    "        # hidden_with_time_axis 的形状 == （批大小，1，隐藏层大小）\n",
    "        # 这样做是为了执行加法以计算分数\n",
    "        hidden_with_time_axis = tf.expand_dims(query, 1)\n",
    "\n",
    "        # 分数的形状 == （批大小，最大长度，1）\n",
    "        # 我们在最后一个轴上得到 1， 因为我们把分数应用于 self.V\n",
    "        # 在应用 self.V 之前，张量的形状是（批大小，最大长度，单位）\n",
    "        score = self.V(\n",
    "            tf.nn.tanh(self.W1(values) + self.W2(hidden_with_time_axis)))\n",
    "\n",
    "        # 注意力权重 （attention_weights） 的形状 == （批大小，最大长度，1）\n",
    "        attention_weights = tf.nn.softmax(score, axis=1)\n",
    "\n",
    "        # 上下文向量 （context_vector） 求和之后的形状 == （批大小，隐藏层大小）\n",
    "        context_vector = attention_weights * values\n",
    "        context_vector = tf.reduce_sum(context_vector, axis=1)\n",
    "\n",
    "        return context_vector, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "k534zTHiDjQU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attention result shape: (batch size, units) (64, 1024)\n",
      "Attention weights shape: (batch_size, sequence_length, 1) (64, 16, 1)\n"
     ]
    }
   ],
   "source": [
    "attention_layer = BahdanauAttention(10)\n",
    "attention_result, attention_weights = attention_layer(sample_hidden, sample_output)\n",
    "\n",
    "print(\"Attention result shape: (batch size, units) {}\".format(attention_result.shape))\n",
    "print(\"Attention weights shape: (batch_size, sequence_length, 1) {}\".format(attention_weights.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yJ_B3mhW3jFk"
   },
   "outputs": [],
   "source": [
    "class Decoder(tf.keras.Model):\n",
    "    def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz):\n",
    "        super(Decoder, self).__init__()\n",
    "        self.batch_sz = batch_sz\n",
    "        self.dec_units = dec_units\n",
    "        self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "        self.gru = tf.keras.layers.GRU(self.dec_units,\n",
    "                                       return_sequences=True,\n",
    "                                       return_state=True,\n",
    "                                       recurrent_initializer='glorot_uniform')\n",
    "        self.fc = tf.keras.layers.Dense(vocab_size)\n",
    "\n",
    "        # 用于注意力\n",
    "        self.attention = BahdanauAttention(self.dec_units)\n",
    "\n",
    "    def call(self, x, hidden, enc_output):\n",
    "        # 编码器输出 （enc_output） 的形状 == （批大小，最大长度，隐藏层大小）\n",
    "        context_vector, attention_weights = self.attention(hidden, enc_output)\n",
    "\n",
    "        # x 在通过嵌入层后的形状 == （批大小，1，嵌入维度）\n",
    "        x = self.embedding(x)\n",
    "\n",
    "        # x 在拼接 （concatenation） 后的形状 == （批大小，1，嵌入维度 + 隐藏层大小）\n",
    "        x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1)\n",
    "\n",
    "        # 将合并后的向量传送到 GRU\n",
    "        output, state = self.gru(x)\n",
    "\n",
    "        # 输出的形状 == （批大小 * 1，隐藏层大小）\n",
    "        output = tf.reshape(output, (-1, output.shape[2]))\n",
    "\n",
    "        # 输出的形状 == （批大小，vocab）\n",
    "        x = self.fc(output)\n",
    "\n",
    "        return x, state, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "P5UY8wko3jFp"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Decoder output shape: (batch_size, vocab size) (64, 4935)\n"
     ]
    }
   ],
   "source": [
    "decoder = Decoder(vocab_tar_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "sample_decoder_output, _, _ = decoder(tf.random.uniform((64, 1)),\n",
    "                                      sample_hidden, sample_output)\n",
    "\n",
    "print ('Decoder output shape: (batch_size, vocab size) {}'.format(sample_decoder_output.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_ch_71VbIRfK"
   },
   "source": [
    "## 定义优化器和损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WmTHr5iV3jFr"
   },
   "outputs": [],
   "source": [
    "optimizer = tf.keras.optimizers.Adam()\n",
    "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True,\n",
    "                                                            reduction='none')\n",
    "\n",
    "\n",
    "def loss_function(real, pred):\n",
    "    mask = tf.math.logical_not(tf.math.equal(real, 0))\n",
    "    loss_ = loss_object(real, pred)\n",
    "\n",
    "    mask = tf.cast(mask, dtype=loss_.dtype)\n",
    "    loss_ *= mask\n",
    "\n",
    "    return tf.reduce_mean(loss_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DMVWzzsfNl4e"
   },
   "source": [
    "## 检查点（基于对象保存）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Zj8bXQTgNwrF"
   },
   "outputs": [],
   "source": [
    "checkpoint_dir = './training_checkpoints'\n",
    "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt\")\n",
    "checkpoint = tf.train.Checkpoint(optimizer=optimizer,\n",
    "                                 encoder=encoder,\n",
    "                                 decoder=decoder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hpObfY22IddU"
   },
   "source": [
    "## 训练\n",
    "\n",
    "1. 将 *输入* 传送至 *编码器*，编码器返回 *编码器输出* 和 *编码器隐藏层状态*。\n",
    "2. 将编码器输出、编码器隐藏层状态和解码器输入（即 *开始标记*）传送至解码器。\n",
    "3. 解码器返回 *预测* 和 *解码器隐藏层状态*。\n",
    "4. 解码器隐藏层状态被传送回模型，预测被用于计算损失。\n",
    "5. 使用 *教师强制 （teacher forcing）* 决定解码器的下一个输入。\n",
    "6. *教师强制* 是将 *目标词* 作为 *下一个输入* 传送至解码器的技术。\n",
    "7. 最后一步是计算梯度，并将其应用于优化器和反向传播。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sC9ArXSsVfqn"
   },
   "outputs": [],
   "source": [
    "@tf.function\n",
    "def train_step(inp, targ, enc_hidden):\n",
    "    loss = 0\n",
    "\n",
    "    with tf.GradientTape() as tape:\n",
    "        enc_output, enc_hidden = encoder(inp, enc_hidden)\n",
    "\n",
    "        dec_hidden = enc_hidden\n",
    "\n",
    "        dec_input = tf.expand_dims([targ_lang.word_index['<start>']] *\n",
    "                                   BATCH_SIZE, 1)\n",
    "\n",
    "        # 教师强制 - 将目标词作为下一个输入\n",
    "        for t in range(1, targ.shape[1]):\n",
    "            # 将编码器输出 （enc_output） 传送至解码器\n",
    "            predictions, dec_hidden, _ = decoder(dec_input, dec_hidden,\n",
    "                                                 enc_output)\n",
    "\n",
    "            loss += loss_function(targ[:, t], predictions)\n",
    "\n",
    "            # 使用教师强制\n",
    "            dec_input = tf.expand_dims(targ[:, t], 1)\n",
    "\n",
    "    batch_loss = (loss / int(targ.shape[1]))\n",
    "\n",
    "    variables = encoder.trainable_variables + decoder.trainable_variables\n",
    "\n",
    "    gradients = tape.gradient(loss, variables)\n",
    "\n",
    "    optimizer.apply_gradients(zip(gradients, variables))\n",
    "\n",
    "    return batch_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ddefjBMa3jF0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 Batch 0 Loss 4.6023\n",
      "Epoch 1 Batch 100 Loss 2.1158\n",
      "Epoch 1 Batch 200 Loss 1.8639\n",
      "Epoch 1 Batch 300 Loss 1.7150\n",
      "Epoch 1 Loss 2.0167\n",
      "Time taken for 1 epoch 943.3117020130157 sec\n",
      "\n",
      "Epoch 2 Batch 0 Loss 1.4880\n",
      "Epoch 2 Batch 100 Loss 1.4871\n",
      "Epoch 2 Batch 200 Loss 1.2971\n",
      "Epoch 2 Batch 300 Loss 1.1837\n",
      "Epoch 2 Loss 1.3627\n",
      "Time taken for 1 epoch 958.1314051151276 sec\n",
      "\n",
      "Epoch 3 Batch 0 Loss 1.0060\n",
      "Epoch 3 Batch 100 Loss 1.0004\n",
      "Epoch 3 Batch 200 Loss 0.8758\n",
      "Epoch 3 Batch 300 Loss 0.8301\n",
      "Epoch 3 Loss 0.9356\n",
      "Time taken for 1 epoch 1042.2323021888733 sec\n",
      "\n",
      "Epoch 4 Batch 0 Loss 0.6176\n",
      "Epoch 4 Batch 100 Loss 0.6259\n",
      "Epoch 4 Batch 200 Loss 0.5790\n",
      "Epoch 4 Batch 300 Loss 0.5839\n",
      "Epoch 4 Loss 0.6183\n",
      "Time taken for 1 epoch 1280.2978203296661 sec\n",
      "\n",
      "Epoch 5 Batch 0 Loss 0.3799\n",
      "Epoch 5 Batch 100 Loss 0.4281\n",
      "Epoch 5 Batch 200 Loss 0.4249\n",
      "Epoch 5 Batch 300 Loss 0.4342\n",
      "Epoch 5 Loss 0.4130\n",
      "Time taken for 1 epoch 1320.4021170139313 sec\n",
      "\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 5\n",
    "\n",
    "for epoch in range(EPOCHS):\n",
    "    start = time.time()\n",
    "\n",
    "    enc_hidden = encoder.initialize_hidden_state()\n",
    "    total_loss = 0\n",
    "\n",
    "    for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):\n",
    "        batch_loss = train_step(inp, targ, enc_hidden)\n",
    "        total_loss += batch_loss\n",
    "\n",
    "        if batch % 100 == 0:\n",
    "            print('Epoch {} Batch {} Loss {:.4f}'.format(\n",
    "                epoch + 1, batch, batch_loss.numpy()))\n",
    "    # 每 2 个周期（epoch），保存（检查点）一次模型\n",
    "    if (epoch + 1) % 2 == 0:\n",
    "        checkpoint.save(file_prefix=checkpoint_prefix)\n",
    "\n",
    "    print('Epoch {} Loss {:.4f}'.format(epoch + 1,\n",
    "                                        total_loss / steps_per_epoch))\n",
    "    print('Time taken for 1 epoch {} sec\\n'.format(time.time() - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mU3Ce8M6I3rz"
   },
   "source": [
    "## 翻译\n",
    "\n",
    "* 评估函数类似于训练循环，不同之处在于在这里我们不使用 *教师强制*。每个时间步的解码器输入是其先前的预测、隐藏层状态和编码器输出。\n",
    "* 当模型预测 *结束标记* 时停止预测。\n",
    "* 存储 *每个时间步的注意力权重*。\n",
    "\n",
    "请注意：对于一个输入，编码器输出仅计算一次。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EbQpyYs13jF_"
   },
   "outputs": [],
   "source": [
    "def evaluate(sentence):\n",
    "    attention_plot = np.zeros((max_length_targ, max_length_inp))\n",
    "\n",
    "    sentence = preprocess_sentence(sentence)\n",
    "\n",
    "    inputs = [inp_lang.word_index[i] for i in sentence.split(' ')]\n",
    "    inputs = tf.keras.preprocessing.sequence.pad_sequences([inputs],\n",
    "                                                           maxlen=max_length_inp,\n",
    "                                                           padding='post')\n",
    "    inputs = tf.convert_to_tensor(inputs)\n",
    "\n",
    "    result = ''\n",
    "\n",
    "    hidden = [tf.zeros((1, units))]\n",
    "    enc_out, enc_hidden = encoder(inputs, hidden)\n",
    "\n",
    "    dec_hidden = enc_hidden\n",
    "    dec_input = tf.expand_dims([targ_lang.word_index['<start>']], 0)\n",
    "\n",
    "    for t in range(max_length_targ):\n",
    "        predictions, dec_hidden, attention_weights = decoder(dec_input,\n",
    "                                                             dec_hidden,\n",
    "                                                             enc_out)\n",
    "\n",
    "        # 存储注意力权重以便后面制图\n",
    "        attention_weights = tf.reshape(attention_weights, (-1, ))\n",
    "        attention_plot[t] = attention_weights.numpy()\n",
    "\n",
    "        predicted_id = tf.argmax(predictions[0]).numpy()\n",
    "\n",
    "        result += targ_lang.index_word[predicted_id] + ' '\n",
    "\n",
    "        if targ_lang.index_word[predicted_id] == '<end>':\n",
    "            return result, sentence, attention_plot\n",
    "\n",
    "        # 预测的 ID 被输送回模型\n",
    "        dec_input = tf.expand_dims([predicted_id], 0)\n",
    "\n",
    "    return result, sentence, attention_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "s5hQWlbN3jGF"
   },
   "outputs": [],
   "source": [
    "# 注意力权重制图函数\n",
    "def plot_attention(attention, sentence, predicted_sentence):\n",
    "    fig = plt.figure(figsize=(10,10))\n",
    "    ax = fig.add_subplot(1, 1, 1)\n",
    "    ax.matshow(attention, cmap='viridis')\n",
    "\n",
    "    fontdict = {'fontsize': 14}\n",
    "\n",
    "    ax.set_xticklabels([''] + sentence, fontdict=fontdict, rotation=90)\n",
    "    ax.set_yticklabels([''] + predicted_sentence, fontdict=fontdict)\n",
    "\n",
    "    ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "    ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sl9zUHzg3jGI"
   },
   "outputs": [],
   "source": [
    "def translate(sentence):\n",
    "    result, sentence, attention_plot = evaluate(sentence)\n",
    "\n",
    "    print('Input: %s' % (sentence))\n",
    "    print('Predicted translation: {}'.format(result))\n",
    "\n",
    "    attention_plot = attention_plot[:len(result.split(' ')), :len(sentence.split(' '))]\n",
    "    plot_attention(attention_plot, sentence.split(' '), result.split(' '))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n250XbnjOaqP"
   },
   "source": [
    "## 恢复最新的检查点并验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UJpT9D5_OgP6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x1a3d2bf320>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 恢复检查点目录 （checkpoint_dir） 中最新的检查点\n",
    "checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WrAM0FDomq3E"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> hace mucho frio aqui . <end>\n",
      "Predicted translation: it s very cold ! <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAAJwCAYAAAA5n02CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xm4ZXdZ5+3vk1QGAwQaghBRJAgYwpyUzAo0tukGRUVFMUxiE1RoQHBC2gbpNyAIKjbaElSQITbDC40ERRkNymRABQwQYkiYTEI0QIqQ+ek/1i45OTkVUqcqZ/32rvu+rrqufdbetes5K5Xan7PG6u4AADCe/eYeAACAjQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQm1AVXVbavqnVV1p7lnAQDmI9TG9Ogk90/y2JnnAABmVG7KPpaqqiRnJXlbkh9I8i3dfcWsQwEAs7BFbTwPSHKDJE9KcnmSB807DgAwF6E2nkcleX13X5TkTzPtBgUA9kF2fQ6kqq6X5F+SPLi731NVd03yvky7Py+YdzoAYKvZojaWH0lyfne/J0m6+x+SfCrJT8w6FQAskaq6XlU9qqpuOPcse0qojeWRSV61btmrYvcnAOyOhyV5WabP1aVm1+cgqurbknw6ye27+1Nrln9rprNAj+ru02cajxVUVXdO8gtJjkrSSU5L8oLu/uisgwHsoap6d5JvTnJRd2+feZw9ItRgH1RVD0nyhiTvSfI3i8X3Xfx6aHe/ea7ZAPZEVd0qyelJ7p7k/UmO7u7T5pxpTwi1gVTVLZN8tjf4j1JVt+zuz8wwFiuoqj6S5I3d/cx1y5+d5Ae7+y7zTAawZ6rq15Lcv7sfWFVvSPKp7v7luefaLMeojeXTSW66fmFV3WTxHOwtt0vyyg2WvzLJd27xLAB706Py9X/fXpXkuMXF5JeSUBtLZTpWaL3rJ7l4i2dhtZ2X5JgNlh+T5NwtngVgr6iqeyc5PMnrFotOTnJIku+dbag9tG3uAUiq6ncXDzvJc6vqojVP759pP/s/bPlgrLKXJnlJVd0myXsz/d27b6aTC35zzsEA9sCjk7ypu7+aJN19aVW9NsljMt2acek4Rm0AVfWuxcP7ZbrA7aVrnr4001mfL1h7NijsicVugKckeVqSb1ks/kKmSPvdjY6TBBhZVR2U5JwkD+/ut65Zft8kf5nkZt29Y675NkuoDWLxwfnaJI/t7gvnnod9R1XdIEn8vQOWWVUdlun+2K9c/8NmVT0iydu7+5xZhtsDQm0QVbV/puPQ7rLMpxEDAHuPY9QG0d1XVNXZSQ6cexZWX1XdOMkJSR6Y6aKQVzmxqLsPnWMuAK5KqI3lfyb5jap6RHefP/cwrLQ/SnK3JCdmOjbNpnVgKVXVp3Mt/w3r7ltfx+PsdXZ9DqSqPprkiCQHJPlckq+ufb677zzHXKyeqvpKkv/U3R+YexaAPVFVT1vz5fWTPDXJBzOdnJck98p09YQXdvezt3i8PWaL2lheP/cA7DPOS7J0Zz8BrNfdL9z5uKpenuR53f2cta+pqqcnucMWj7ZX2KIG+6Cq+vEkD0vy6GU8XR1gI4u9BUd39xnrlt8myYeX8fhbW9RYCVX1c0mekGnX8R27+8yq+pUkZ3b3a+edbgyLXetrfzI7Isl5i5NYLlv7WrvZgSX11ST3T3LGuuX3T3LR+hcvA6E2kKo6MMkzkjw8yS0zHav277p7/znmGl1VPSXJLyV5XpLfWPPU55M8MdP16bBrHVh9v53k96pqe5L3L5bdM9MdC54111B7wq7PgVTV85L8eJLnZvrL9t+T3CrJTyT5te5+yXzTjauqPpHkad39lqq6MNO16M6sqjskOaW7bzLziLBPqaqjk/xDd1+5eLxL3f3hLRqLfURVPSzJk5PcfrHo40letKx7V4TaQBanGP9sd791ERx37e5/rqqfTfLA7v7RmUccUlV9LcmR3X32ulC7XaYPi0NmHnE4VXW/JOnuv95geXf3KbMMxkqoqiuT3Ly7z1s87iS1wUvbngK4ZnZ9juVmSXbelWBHkhstHr810249NnZmkqOTnL1u+YPy9fXJVf12ko1OUz800+6BY7Z0GlbNEUm+uOYxbLmqulGufjHvf5tpnE0TamP5TKYbZH8m04GQxyb5UKZrwHxtxrlG94IkL66qQzL91H6vqnpkpuPWHjvrZOP6ziT/uMHyjy6eg03r7rM3egzXtar69iR/kOQBuepx3pVpy+7SbcEVamN5Y6Zb+rw/yYuS/GlVPS7JLZL85pyDjay7X1ZV25I8J8khSV6Z6USCJ3X3a2Ydblxfy/RDwafXLf/WJJdu/TisKseoscVelmlv1GOzInddcYzawKrqHknuk+T07j557nmWQVUdlmS/7j5v7llGVlWvznRm8UO6+4LFshsn+b9JPt/dD59zPlbHLo5R+/cPHseosTdV1Y4k9+zuj809y94i1AZSVd+T5L3dffm65duS3NsB3htbnN25f3d/ZN3yOye5vLsdp7ZOVR2e5JRMN2Tfud7unOmOBffr7i/MNRurZbEraq0DMt1n9hlJnt7df7H1U7GqFteLfEx3f2juWfYWoTaQqroiyeHrtwZV1U2SnOcnz41V1d8m+b3uPmnd8p9I8sTuvu88k41tcUzfcUnummlrx4eTnNTdS3lRyK1SVf8xyVGZtgqd1t3vmnmkpVRV35fkmd19n7lnYXUs/v/8lSQ/t/7uBMtKqA1ksYvgZt39xXXLb5fk1GW89cVWWFyS424b3DLkOzLdMuSG80zGKqmqW2Q6jvSYTMe+JNNxfqcm+WFbIXdPVd020+Vzrjf3LKyOxefBQZlOGrgkyVX2UC3j56iTCQZQVX+2eNhJXlVVl6x5ev8kd0zy3i0fbHlckWSjGPsP2fjaTfu8qnroNT3f3W/YqlmWyO9m+rt2m+7+dJJU1a2TvGrxnOscbmBx7ONVFiU5PNNlYD655QOx6p449wB7my1qA6iqly0ePjrT7Y7WXorj0iRnJXlpd5+/xaMthap6U6YP0B/r7isWy7YleV2SA7r7++ecb0SLrbcb6cQB3htZ3Oz5/uvPUlzcquYdttxubM3JBFdZnOSzSX68u99/9d8F7GSL2gC6+6eSpKrOSvKC7v7qvBMtnV9K8jdJzqiqv1ksu2+S6yf5ntmmGlh3X+UikIuwvVumy8A8Y5ahlteuopfJA9Z9fWWmi+Gesf7EKdgbqupmSR6Z5Dsy3X7x/Kq6T5Iv7NwavkxsURtIVe2XJN195eLrmyf5/kwHLNv1eQ0WZzE+MVc9MP73HTe0e6rq3kn+d3ffZe5ZRlNVb0xy0yQP7+7PLpbdMsmrk3yxu69xdzJw3auqY5K8I9M1Iu+Q6faCZ1bVs5Lcrrt/cs75NkOoDaSq/iLJW7v7RVV1/SSfSHK9TFuGfrq7XzHrgKy8qjoqyQe7+/pzzzKaqvq2JG9Kcqd8/UKat8h0eZMf7O7PzTjesBaXHbpWXIKIPVVV70pySnc/c929n++V5P909/rLxQzPrs+xHJNpN16SPDTJVzLdJ++4JL+QRKhdg6r6lkwXcT1w7XL/+F/dBleL33mA9y8n+futn2h8i61oR1fVf0pyZKZ1dlp3v33eyYb37nz9GLWdJ/es/3rnMsdGsqeOSfLTGyz/l0z30146Qm0sN0jypcXj70vyxu6+rKremeT35htrbItAOynT8Wg7r4C+dlOxf/yv7tRc/WrxyXT7MvdHvQbd/bYkb5t7jiXy/Znux3tCkvctlt0rya9m+sHUyQTsTV/LdMb/ekdmuqD30hFqY/lMkvtU1Zsz3ZD9xxbLb5zERUh37XcynfV5VJK/S/KfM/3k9OwkPz/jXCM7Yt3XV2Y6zuriOYYZVVU9NdOxjhcvHu9Sd//WFo21bP5nkicvAnenM6vqvCTP7+67zTQXq+lNSZ5ZVTs/P7uqbpXkeUn+/7mG2hOOURtIVT0+yYuT7EhydpKju/vKqnpSkh/q7v8464CDqqpzkzy4u09dXEJhe3efXlUPznTGzz1nHnFIi5NV7p3pNlJXOQu0u39/lqEGU1WfzvT36V8Xj3elu/vWWzXXMqmqr2X6t+zj65YfleRD3f1N80zGKqqqQ5P8eaZb4l0vyTmZfnB/b5L/soxXVRBqg1mcsXLLJG/r7h2LZQ9O8qXu/ttZhxvUIs7u3N1nLS5x8oju/puqOiLJP3X3IfNOOJ6qekSSP8y06/OCXHVXcXf3t8wyGCunqk5NckaSn+rury2WfVOSl2W6ePD2OedjNS1uJXV0ph9CP7zMx5La9TmIqrphpth4T5L1N5P9UhI3Ft+1T2Q6/uCsJP+Q5Geq6rNJnpDk8zPONbITkjw/ybNdy+obq6oDMl2r71Hd7Wr6u+dnk5yc5PNV9ZHFsjtlOlzhwbNNxcpZ+zna3e9M8s41z90n08k/F8w24CbZojaIqrpBprNSjl275ayq7prkA0lu4c4EG6uq4zLdgeDli7MZ35rksEz3eXt0d7921gEHVFUXJDmmu8+ce5ZlsTim6r7dffrcsyybqjok09nrt8/ibNkkJy3jbijGtaqfo0JtIFX16iQ7uvvxa5a9INNF+h4y32TLZfGhcGSSzyzj/5RboapenOST3f2/5p5lWVTVbyZJd//i3LMsm8WdL+6ejS+f47JD7DWr+Dkq1AZSVccm+dMkN1tclmO/JJ9L8kQ3yb5mVfXjSR6YjQ+MX8r/Oa9LVXVgkv+b6V6yH01y2drnu/vZc8w1sqr6/UxbhT6d6fCEq2wN6u4nzTHX6KrqyCRvznSmcWXa5bkt09+5S7r70BnHY8Ws4ueoY9TG8rZMl+H4gSRvyBQeB2b6R45dWGzpeEqSd+XrV4znmj0+02VMzk9ym6w7mSDTpU32eYur6r93cRzf7TPdmixJ1p/h6e/crv1OprC9a6Yz8O6a5IZJ/neS/z7jXKymlfsctUVtMFX1vCTf2d0/VFWvSHJhdz9h7rlGtrg8xxO6+/Vzz7IsFsdbPbe7f3vuWUZWVVckOby7z6uqM5N8V3f/69xzLZOq+tck9+vuj1XVl5Pcvbs/WVX3S/K/uvvOM4/Iilm1z1Fb1MbziiQfWtxX8Icz/TTANdsv09meXHv7J/mzuYdYAhdk2mV3XpJbZd1uda6Vytcv2P3FTPdH/WSm3VG3mWsoVtpKfY7aojagqvq7JBcnOay7bz/3PKOrqhOSXNbdz5p7lmWxOLj2K45Fu2ZV9ZIkj850JtktM8XFFRu91gVvN1ZVpyT57e5+Y1WdlOQmSZ6T5HGZLqVgixp73Sp9jtqiNqZXZjqu4xlzDzKqqvrdNV/ul+S4xc2yP5KrHxjvIO+rOyTJf10ceGud7drPZNryeNskv5XpIq0XzjrR8jkh0xXik+mYtJMzHU96fpKHzTXUMquqjye5bXf7DN+1lfkc9R95TK/KdFPZl809yMDutO7rnbs+j1y33Cbjjd0+yd8vHltnu9DTLoe3JElV3SXJC7tbqO2G7v7LNY/PTHJUVd04yQVtl85m/V6mLZPs2sp8jtr1CQAwKAfGAgAMSqgBAAxKqA2sqo6fe4ZlZL3tPutsc6y3zbHedp91tjmrsN6E2tiW/i/YTKy33WedbY71tjnW2+6zzjZn6debUAMAGNQ+f9bngXVQH/zvl/gZy2W5JAfkoLnHWDrW2+6zzjbHetsc6233WWebM/J6uzAXnN/dN/1Gr9vnr6N2cK6Xe9RS310CAFgyb+/Xn31tXmfXJwDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCgViLUqurlVXXy3HMAAOxN2+YeYC95cpJKkqp6d5KPdfcTZ50IAGAPrUSodfeX554BAGBvW4lQq6qXJzksyflJ7pfkflX1hMXTR3T3WTONBgCwaSsRams8Ocntknwiya8uln1xvnEAADZvpUKtu79cVZcmuai7z9nV66rq+CTHJ8nBOWSrxgMA2C0rcdbn7uruE7t7e3dvPyAHzT0OAMCG9slQAwBYBqsYapcm2X/uIQAA9tQqhtpZSe5eVbeqqsOqahW/RwBgH7CKEfOCTFvVTst0xuct5x0HAGBzVuKsz+5+zJrHpye513zTAADsHau4RQ0AYCUINQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEFtm3uAudW2/bP/jW489xhL565vP3/uEZbOB39++9wjLKUD3n/a3CMspb788rlHWDrWGSOyRQ0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQKxdqVfU9VfX+qtpRVV+uqg9U1R3nngsAYHdtm3uAvamqtiV5U5I/SnJckgOSHJ3kijnnAgDYjJUKtSSHJrlRkjd39z8vln1i/Yuq6vgkxyfJwftdf+umAwDYDSu167O7/y3Jy5P8ZVW9paqeWlXftsHrTuzu7d29/cD9Dt7yOQEAro2VCrUk6e6fSnKPJKckeUiS06vq2HmnAgDYfSsXaknS3f/Y3c/r7vsneXeSR887EQDA7lupUKuqI6rqN6rq3lX17VX1gCR3TnLa3LMBAOyuVTuZ4KIkt0vyuiSHJTk3yauTPG/OoQAANmOlQq27z03y0LnnAADYG1Zq1ycAwCoRagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIPaNvcAs7uy0xdfMvcUS+cff+TWc4+wdG70J5+Ze4SltOOpt517hKW03yfPnnuEpXPFjq/OPcJyuvKKuSdYabaoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADGrpQ62qDpx7BgCA68KWhlpVPb6qzq2qbeuWn1RVb1o8/oGq+lBVXVxVn66qE9bGWFWdVVXPqqo/rqovJXl1Vb2zql687j0PraqLquqhW/LNAQDsZVu9Re21SW6U5Ht3Lqiq6yX5wSSvqqpjk7w6yYuT3CHJY5P8aJLnrHufpyb5RJLtSX41yUuT/GRVHbTmNQ9PsiPJm6+T7wQA4Dq2paHW3Rck+fMkx61Z/MNJLs8UVM9I8pvd/bLu/ufufleSX07yM1VVa37PX3f387v7jO7+VJI3JLly8V47PTbJK7r7svVzVNXxVXVqVZ16aV+8V79HAIC9ZY5j1F6V5Ieq6pDF18cleX13X5zkmCTPqKodO38lOSnJ9ZLcfM17nLr2Dbv7kiSvzBRnqaqjktw9yR9vNEB3n9jd27t7+4F18F781gAA9p5t3/gle93Jmbag/WBVvSPTbtDvWzy3X5JfT/K6DX7fF9c8/uoGz/9hko9U1S2T/HSS93X3aXttagCALbblodbdl1TV6zNtSTssyTlJ/nrx9IeTHNndZ2ziff+pqj6Q5HFJHpFpNyoAwNKaY4taMu3+fHuSI5Kc1N1XLpY/O8nJVXV2phMPLk9yxyR37+5fuhbv+9Ikf5DksiSv2etTAwBsobmuo3ZKks8nOSpTtCVJuvsvkzw4yQOSfHDx61eSfOZavu9rklya5LXdfeHeHBgAYKvNskWtuzvJrXbx3F8l+atr+L0b/r6FGyX5piR/tAfjAQAMYa5dn3tVVR2Q5PAkJyT5++7+25lHAgDYY0t/C6mF+yQ5O8k9Mp1MAACw9FZii1p3vztJfaPXAQAsk1XZogYAsHKEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKC2zT3A3PrKK3PlRRfNPcbSufLMs+YeYemc/5zvmnuEpXTOL1489whL6WavOHLuEZbOwW/5u7lHgKuxRQ0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBCDQBgUEINAGBQQg0AYFBLGWpV9ayq+tg3eM2Lq+rdWzQSAMBet5ShBgCwLxBqAACDmi3UavK0qvpUVV1SVZ+rqucunrtTVb29qr5WVf9WVS+vqhtew3vtX1UvqKoLFr9+J8n+W/bNAABcB+bcovacJL+W5LlJ7pDkx5J8tqoOSfLWJDuS3D3JDye5d5I/vob3elqSxyV5fJJ7ZYq0466zyQEAtsC2Of7Qqrp+kp9P8pTu3hlgZyR5X1U9Lsn1kzyyuy9cvP74JO+qqtt09xkbvOVTkjy/u1+7eP2Tkxx7DX/+8UmOT5KDc8he+q4AAPauubaoHZXkoCTv2OC52yf5yM5IW3hvkisXv+8qFrtED0/yvp3LuvvKJB/Y1R/e3Sd29/bu3n5ADtrcdwAAcB2bK9TqGzzXu3huV8sBAFbOXKF2WpJLkjxwF8/dpapusGbZvTPN+vH1L+7uLyf5lyT33LmsqirT8W0AAEtrlmPUuvvCqnpRkudW1SVJTklykyTHJPmTJL+e5BVV9T+S/IckL0nyhl0cn5YkL0ry9Ko6PclHk/xcpt2h/3LdficAANedWUJt4elJLsh05ue3Jjk3ySu6+6KqOjbJ7yT5YJKLk7wpyZOv4b1emOTmSf5w8fUrk7w60/FuAABLabZQWxzw/xuLX+uf+2g23i268/lnJXnWmq8vz3QW6c/v7TkBAObizgQAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIPaNvcAQ+ieewL2AQf9xd/NPcJSutXbD5x7hKX01rM/OPcIS+fBRx879whL6fJzzp17hJVmixoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKBWNtSq6qyq+oW55wAA2KyVDTUAgGUn1AAABiXUAAAGtW3uAa5DVy5+XU1VHZ/k+CQ5OIds5UwAANfaKm9R27H4dTXdfWJ3b+/u7QfkoC0eCwDg2lnlUPtydhFqAADLYGV3fXb3d889AwDAnljZLWpV9Y6q+qm55wAA2KyVDbUk35HkJnMPAQCwWau86/NWc88AALAnVnmLGgDAUhNqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAgxJqAACDEmoAAIMSagAAg9o29wAA16Qvv2zuEZbSg+7/I3OPsHQOfM2OuUdYSl99/nfNPcJyOvn11+pltqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxKqAEADEqoAQAMSqgBAAxqaUKtqn6hqs6aew4AgK2yNKEGALCv2SuhVlWHVtWN9sZ77cafedOqOngr/0wAgK206VCrqv2r6tiqOinJOUnuslh+w6o6sarOq6oLq+qvq2r7mt/3mKraUVUPrKqPVdVXq+pdVXXEuvf/pao6Z/HaVyS5/roRHpTknMWfdZ/Nfh8AAKPa7VCrqjtU1fMp1pnCAAAFuklEQVSTfCbJa5J8Ncl/TnJKVVWStyS5RZLvT3K3JKckeWdVHb7mbQ5K8vQkj01yryQ3SvIHa/6MhyX5/5I8M8nRST6Z5KnrRnlVkp9McoMkb6uqM6rqf6wPPgCAZXWtQq2qblJVT6qqU5P8fZIjkzwlyc26+3HdfUp3d5IHJLlrkh/t7g929xnd/WtJzkzyyDVvuS3JExav+UiSFyR5QFXtnOcpSf6ku1/S3ad39wlJPrh2pu6+orv/vLsfnuRmSZ6z+PM/tdiK99iqWr8Vbuf3c3xVnVpVp16WS67NKgAA2HLXdovaf0vyoiSXJLltdz+ku1/X3esr55gkhyT54mKX5Y6q2pHkjkm+Y83rLunuT675+gtJDsi0ZS1Jbp/kfevee/3X/667L+zuP+7uByT5riTfnOSPkvzoLl5/Yndv7+7tB+Sga/i2AQDms+1avu7EJJcleVSSf6qqNyZ5ZZJ3dPcVa163X5Jzk3z3Bu/xlTWPL1/3XK/5/butqg5K8uBMW+0elOSfMm2Ve9Nm3g8AYATXKoy6+wvdfUJ3f2eS702yI8n/SfK5qnphVd1t8dIPZ9oNeeVit+faX+ftxlwfT3LPdcuu8nVN7ltVL8l0MsOLk5yR5JjuPrq7X9TdF+zGnwkAMJTd3oLV3e/v7p9NcnimXaK3S/LBqvruJG9P8rdJ3lRV/6Wqjqiqe1XVry+ev7ZelOTRVfW4qrptVT09yT3WveYRSf4qyaFJHp7k27r7F7v7Y7v7PQEAjOja7vq8msXxaa9P8vqq+uYkV3R3V9WDMp2x+dJMx4qdmyneXrEb7/2aqrp1khMyHfP2Z0l+K8lj1rzsHUlu3t1fufo7AAAsv5pO1tx3HVo37nvUA+ceA9iVqrknWEr73/bWc4+wdA586Y65R1hKX3r+LeceYSm95+Rf/lB3b/9Gr3MLKQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQW2bewCAa9Q99wRL6YrT/3nuEZbO1+439wTL6aCcO/cIK80WNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBCTUAgEEJNQCAQQk1AIBBbZt7gDlU1fFJjk+Sg3PIzNMAAGxsn9yi1t0ndvf27t5+QA6aexwAgA3tk6EGALAMhBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCghBoAwKCEGgDAoIQaAMCgqrvnnmFWVfXFJGfPPccuHJbk/LmHWELW2+6zzjbHetsc6233WWebM/J6+/buvuk3etE+H2ojq6pTu3v73HMsG+tt91lnm2O9bY71tvuss81ZhfVm1ycAwKCEGgDAoITa2E6ce4AlZb3tPutsc6y3zbHedp91tjlLv94cowYAMChb1AAABiXUAAAGJdQAAAYl1AAABiXUAAAG9f8AXLDDCrV9jncAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'hace mucho frio aqui.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zSx2iM36EZQZ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> esta es mi vida . <end>\n",
      "Predicted translation: this is my life . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAAJwCAYAAAA5n02CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xu0bQdZ3/3fQ04uQgi83MMdCxTl2nDkIi0G8RVFZRReaqsEwqWkw1crvlRpGR0opSIFoxZfrCWohJsWzFuLgugLAoXKbUREEFCIQABDuAhCQjDXp3+sdWSzOSecs89lPmvn8xljj7P2XGuv/ew5zjnru+dcc87q7gAAMM/1lh4AAID9E2oAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJtYGq6i5V9caquufSswAAyxFqM52Z5PQkT1x4DgBgQeWi7LNUVSX5WJLXJ/mBJLfu7qsXHQoAWIQtavM8JMkNk/x4kquSPHzZcQCApQi1eR6X5LzuvizJb2W1GxQAuA6y63OQqrpBkk8l+b7ufmtV3SfJ27Pa/fmFZacDAI41W9Rm+b+SfK6735ok3f2eJB9O8i8WnQoANkhV3aCqHldVN1p6lsMl1GZ5bJKXb1v28tj9CQCH4geTvDir19WNZtfnEFV1uyQfTfIt3f3hLctvm9VRoN/a3R9aaDwA2BhV9eYkt0hyWXfvXXicwyLUAIBdo6rumORDSe6X5B1JTuvuDyw50+Gw63OQqrr9+jxq+73vWM8DABvosUneun6f9+9nw98+JNRm+WiSm29fWFU3Xd8HAFy7xyV52fr2y5M85kAbQTaBUJulkuxvX/TJSf7uGM8CABulqr49yalJfnu96DVJrp/kuxYb6jDtWXoAkqr65fXNTvKcqrpsy93HZbWf/T3HfDAA2CxnJnl1d385Sbr7iqp6VZLHZ3Vpxo0j1Ga45/rPSvItSa7Yct8VSd6d5OxjPRQAbIqqOjGr03L80La7Xp7kD6vq5O6+9NhPdngc9TnEev/5q5I8sbsvWXoeANgkVXWzrK6P/bLeFjdVdUaSN3T3xYsMdxiE2hBVdVxW70O79yYfRgwAHDkOJhiiu69OcmGSE5aeBQCYwRa1QarqzKz2rZ/R3Z9beh4AmK6qPpr9nzHh63T3Nx/lcY44BxPM8pNJ7pTkr6vqk0m+vPXO7r7XIlMBwFwv2HL75CRPTfKuJG9fL3tgVmdP+IVjPNcRIdRmOW/pAQBgk3T33wdYVZ2b5Lnd/XNbH1NVT09y92M82hFh1ycAsCtU1ZeyurbnBduW3znJu7v7lGUm2zkHEwAAu8WXk5y+n+WnJ7lsP8vHs+tzkKo6Icm/z+qAgtsnOX7r/d193BJzAcCG+KUkv1JVe5O8Y73sAVldseCZSw11OITaLP8xyT9P8pys/rL9VJI7JvkXSZ6x3FgAMF93P6+qPpbkKVldpSBJPpjkzO5+1WKDHQbvURtkfYjxj3T3H1TVJUnu091/VVU/kuSh3f3ohUccqaqekK9uhfya89Bt4qHYsNtU1U2SfE/2/2/0WYsMBRvCFrVZbplk31UJLk1y4/XtP0jy3EUmGq6qfirJ05O8MMmDk/yXJHde33Z9VFhYVT0gyWuTXJ7k5kn+Osmp688/lkSocVRU1Y2z7b343f35hcbZMQcTzPLxJLde374gycPWtx+Y5CuLTDTfk5Oc1d1PT3Jlkhd09yOyOl/OHRadDEiSn0/yiiS3yeoyed+Z1Za18+MXUI6wqrpDVb2uqv4uyd8k+ez643PrPzeOLWqz/E6Sh2b1BsjnJ/mtqnpyVv/B/fySgw1226xObJisYnbfode/tV7+5CWGAv7evZI8qbu7qq5OcmJ3f6Sq/m2S38wq4uBIeXFWe6OemOSiHOQVCyYTaoOstwrtu31eVX0iyYOSfKi7X7PcZKNdnORmWW2NvDCrrY/vyWr358b/A4Vd4Iottz+d1ZbuD2b19o5b7/crYOful+QB3f3nSw9ypAi1QarqwUne1t1XJUl3vzPJO6tqT1U9uLvfsuyEI70xySOSvDvJryf5par6wSSnJdnII3xgl3l3km9L8qEkb07ys1V1yyRnJHnvgnOxO300yYlLD3EkOepzkPVugVO7+zPblt80yWecR+3rVdX1klxvX9xW1T/Peitkkhd295VLzgfXdevzWd2wu99UVTdP8tJ89d/oE7r7fYsOyK5SVd+Z5N8l+b+3X51gUwm1QarqmiS37O7Pblt+1yTnb+KlL462qrp9kk/0tr/IVVVJbtfdH19mMgCOtfWprU5MclxWRxZftfX+TXwdtetzgKr63fXNTvLyqrp8y93HJblHkrcd88E2w0ezOtT/M9uW32R9n62QANcdP7b0AEeaUJvhb9Z/VpIv5GtPxXFFkv+V5EXHeqgNUdn/QQMnZ3UqAOAYW5+8+6B21zgpNUdSd79k6RmONKE2QHc/IUnWl704u7u/vOxE81XVL69vdpLnVNXWi+0el9WRP+855oMBSfKCLbdPTvLUrE6X8/b1sgdm9W/0F47xXFwHrA9WeWySf5DkGd39uap6UJKLuvujy0536LxHbZD1G+PT3desP79Vku9P8oHututzi6p60/rmd2T1n//WUwBckdUZz8/u7g8f49GALarq3KxOMfRz25Y/Pcndu/uMRQZjV6qq+yb5o6ze+nL3JHdbn7fvmUnu2t0/vOR8OyHUBqmq1yX5g+5+flWdnOQvktwgq99In9TdL110wIGq6sVJntLdX1p6FuDrVdWXkpy2/Qi8qrpzkndv4pu7mWv9S/xbuvtn1gcW3Hsdag9M8t+6e+OuWGPX5yz3TfK09e1HJflSkjsleUySn8zqsHa22LfbeJ+q+qasDv3/cHdfuMxUm8d6O7CqelSS3+vuK9e3D6i7//sxGmuTfDnJ6VldFm+r05Nctv3BcJjum+RJ+1n+qayup71xhNosN0zyt+vb353kd9YvDm9M8ivLjTXXerfKu7r7v1TVCVm9D+buSa6oqkd29+sWHXAo6+2QnJfkVlkdWXzetTyu4yjj/fmlJL+yPp/aO9bLHpDkzCTPXGoodq2vJPk/9rP8bvn6swNsBBdln+XjSR5UVTfI6oLsr18vv0n85nkgD8tX//N/RFaxe6usXgCeucxIG8F6O0jdfb19J6Fe3z7Qh0jbj+5+XlZv7L5nkl9cf9wzyZnd7aLsHGmvTvIzVbXv6gRdVXdM8twk/99SQx0O71EbpKr+VVZHS12a1XUrT+vua6rqx5P80+7+zkUHHKiq/i7Jnbv7k1X1a0m+2N3/Zv0P833dfcNFBxzKetu59UE+357kFvnaX3a7u391mamAJKmqU5L8fpJ7ZfUe74uz2uX5tiTfu4lnVbDrc5DufmFVnZ/k9klev+/ozyR/leQZy0022sVJ7lFVn8pqK9FZ6+UnJ3H5qAOz3nagqs5I8mv56jkPt/6m20mEGixofWDZP15fSuq0rH6Zend3v2HZyXZOqA1RVTdKcq/ufmuSP9l2998m+cCxn2oj/EaSVya5KMnVWR2WnST3z+qoWfbPetuZZyd5XpJn7bu+LF9vfaTnN6/PX3VJruXkt4765EjZ+jra3W9M8sYt9z0oq1NdfWGxAXdIqM1xTZLXVdXDuvuP9y2sqvtk9ZftNotNNlh3P6uq/jzJHZK8qrv3nU/tqqzek8B+WG87dkqSc0XaN/Svk1yyvr3rLunDWLvyddTBBEN09yVZvQnycdvuOiPJH3b35479VBvjK0m+K8nrq+p262UnZPVePw7Mejt0r0jyfUsPMV13v6S7912z+J9mFW2/tV7+NR8Ljskus1tfR4XaLC9N8s+q6vjk769U8MNJzl1yqMmq6jFJXpXkQ1mdc+749V3Xy1fPScc21tuOPTXJ91bV/6iq/1hVP731Y+nhhvpKVv+3fbqqXlRVD156IHa1Xfc6KtRmeX1Wp+H4gfXnD81qC8fvLTbRfE9L8uTu/n+y2m23zzuS3GeZkTaC9bYz/yrJ92R11Ocjk/yzLR+PXnCusdaX7LlFVrtDb5PkDVV1YVU9p6ruvux07EK77nVUqA2yPsrzFfnqZtvHJnlldzsK78Dukq9e6HmrS7N6PxH7Z73tzDOS/JvuvkV336O777nl415LDzdVd1/W3S/v7odnFWs/n9UL6Z8tOxm7zW58HXUwwTwvTfIn6/cMPTKr3wY4sIuS3DWr885t9eCsTmvC/llvO3Nckt9deohNVVUnJfnOrE4Jc9ckn1h2InapXfU6aovaMN39/iTvS/KbST7Z3e9aeKTpzknyy+tDr5PkdlV1ZlanUHBOqwOz3nbmxVlde5eDVFXXq6rvrqqXJPl0Vn+/PpXku7r7TstOx260215HbVGb6WVJ/nOSf7/0INN19/PW5855fZKTkrwpyeVJzu5u10c9AOttx66f5F9W1cOSvDfbTg7c3T++yFSzXZTkRklel+QJSV6z5XQw7EBVfTDJXbrba/iB7ZrXUZeQGqiqbpLVG29f2N0XLz3PJqiq6yf51qy2En+gu51i4iBYb4emqt50LXe3y7x9vao6K6tz9f3t0rPsFlX1Y0lu2t3/YelZptpNr6NCDQBgKO9RAwAYSqgBAAwl1AZbv7eDQ2S9HTrrbGest52x3g6ddbYzu2G9CbXZNv4v2EKst0Nnne2M9bYz1tuhs852ZuPXm1ADABjqOn/U5wl1Yp+UGyw9xn5dmctzfE5ceoyNY70dOutsZ6y3nbHeDp11tjOT19sl+cLnuvvm3+hx1/mT5Z2UG+T+tdFXlwAANswb+rztl/DbL7s+AQCGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADDUy1Krq9KrqqrrZ4TwGAGCTjQi1qnpzVb3gEL/sbUlOTfI3R2EkAIDF7Vl6gJ3q7iuSXLz0HAAAR8viW9Sq6twk35HkR9e7MjvJHdd337uq3llVl1XV+VV12pav+5pdn1V1o6p6WVV9pqr+rqo+UlU/cax/HgCAI2XxUEvylCRvT/LirHZlnprkE+v7npPk3yU5LatdnK+oqjrA8/xsknsm+f4kd0vyxCR/ffTGBgA4uhbf9dndX6yqK5Jc1t0XJ0lV3W199zO6+03rZc9K8r+S3CbJJ/fzVHdI8qfd/a715x870PesqrOSnJUkJ+X6R+LHAAA44iZsUbs2791y+6L1n7c4wGN/NckPVtWfVdXZVfUdB3rS7j6nu/d2997jc+KRmhUA4IiaHmpXbrnd6z/3O3N3vy6rrWpnJ7lZktdW1YuP7ngAAEfPlFC7Islxh/sk3f257n5Zdz8+yZOSnFlVNpkBABtp8feorX0syf2q6o5JLs0OAnL9HrZ3J3l/Vj/Xo5J8pLsvP2JTAgAcQ1O2qJ2d1Va1DyT5bJLb7+A5Lk/y7CR/luSPk9wwyQ8cqQEBAI616u5v/Khd7JS6Sd+/Hrr0GADAdcgb+rw/6e693+hxU7aoAQCwjVADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQe5YeYGl1wvHZc+vbLT3Gxvng02699AgbZ++9L1h6hI305Udf5/+b2pFrvnTJ0iNsnL7yqqVH2Eh91ZVLj7CZ+uAeZosaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQGx1qVXVuVb1m6TkAAI6GPUsPcJiekqSWHgIA4GjY6FDr7i8uPQMAwNGya3Z9VtWDq+odVXVpVX2xqt5ZVfdYekYAgJ3a6C1q+1TVniSvTvLrSR6T5PgkpyW5esm5AAAOx64ItSSnJLlxkt/r7r9aL/uLAz24qs5KclaSnHTcDY/+dAAAO7DRuz736e7PJzk3yR9W1Wur6qlVdbtrefw53b23u/eecNw3HbM5AQAOxa4ItSTp7ickuX+StyR5RJIPVdXDlp0KAGDndk2oJUl3/1l3P7e7T0/y5iRnLjsRAMDO7YpQq6o7VdV/qqpvr6o7VNVDktwryQeWng0AYKd2y8EElyW5a5LfTnKzJJ9O8ookz11yKACAw7HRodbdj9/y6aOWmgMA4GjYFbs+AQB2I6EGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AICh9iw9wNL6iitz1YWfWHqMjXPXn/j00iNsnPc+475Lj7CRbnquv2s7ceLZt116hI1z0oVfWHqEjXTNRz++9Aib6ZqDe5gtagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADDUuFCrqjdX1a9W1S9U1eer6rNV9ZSqOrGqfqWq/raqPl5Vj10//o1V9YJtz3FKVV1WVY9a5qcAADh840Jt7TFJLkly/yT/Kcl/TvI/knwoyd4kL0nya1V16yQvSvLDVXXilq//oSSXJvm9Yzk0AMCRNDXU3t/dz+zuDyf5xSSfS3Jldz+/uy9I8qwkleTbk/z3JNckeeSWr39ikpd295X7e/KqOquqzq+q86/M5Uf1BwEA2KmpofbefTe6u5N8Jsn7tiy7MskXktyiuy9P8rKs4ixV9a1J7pfkNw705N19Tnfv7e69x+fEAz0MAGBRe5Ye4AC2bwnrAyzbF5q/luS9VXX7JE9K8vbu/sDRHREA4OiaukXtkHT3+5O8M8mTk5yRa9maBgCwKaZuUduJFyX5r1lteXvlwrMAABy2XbFFbe2VSa5I8qruvmTpYQAADte4LWrdffp+lt1jP8tutW3RjZN8U5JfPzqTAQAcW+NC7VBV1fFJTk3y7CR/2t1/vPBIAABHxG7Y9fmgJBdmdXLcJy88CwDAEbPxW9S6+81ZnfwWAGBX2Q1b1AAAdiWhBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADLVn6QHYTH3lFUuPsHHu8NNvX3qEjXTFw/YuPcJG+szeE5YeYeN8389/ZOkRNtL7/s+bLj3CZvrswT3MFjUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUCNDrarOrarXbL+9/vx6VfXCqvqbquqqOn2xQQEAjqI9Sw9wEJ6SpLZ8/vAkT0hyepKPJPn8AjMBABx140Otu7+4bdGdk3yqu9+2xDwAAMfKyF2fW23fDZrkl5Lcfr3b82Pr5VVVT6uqv6qqr1TV+6rqjOWmBgA4fOO3qG3zlCQXJnlikm9LcvV6+c8meXSSH03yl0kemORFVfWF7n7tEoMCAByujQq17v5iVV2S5OruvjhJquoGSZ6a5Lu7+63rh360qu6XVbh9XahV1VlJzkqSk3L9YzI7AMCh2qhQO4BvTXJSkj+oqt6y/PgkH9vfF3T3OUnOSZJT6ia9v8cAACxtN4TavvfZ/UCSj2+778pjPAsAwBGzG0LtA0kuT3KH7n7j0sMAABwpGx9q3X1JVZ2d5OyqqiRvSXJykgckuWa9mxMAYONsfKitPSPJp5P8ZJJfTfKlJO9J8rwlhwIAOBwjQ627H7+/2+vPz05y9rZlneT/XX8AAOwK4094CwBwXSXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAw1J6lBwC4Nif84flLj7CRbvtH/ns/VJ9/5A2WHmEjXfqKU5YeYTN998E9zBY1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMtWfpAZZQVWclOStJTsr1F54GAGD/rpNb1Lr7nO7e2917j8+JS48DALBf18lQAwDYBEINAGCoXRtqVfVjVfUXS88BALBTuzbUktwsyT9ceggAgJ3ataHW3c/s7lp6DgCAndq1oQYAsOmEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhtqz9AAAHHl91VVLj7BxLnr0rZYeYSO95Z2/s/QIG+m4g3ycLWoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoTYm1KrqJ6vqY0vPAQBwrGxMqAEAXNcckVCrqlOq6sZH4rkO4XvevKpOOpbfEwDgWNpxqFXVcVX1sKr6zSQXJ7n3evmNquqcqvpMVV1SVf+zqvZu+brHV9WlVfXQqvrzqvpyVb2pqu607fmfVlUXrx/70iQnbxvh4UkuXn+vB+305wAAmOqQQ62q7l5Vz0vy8SSvTPLlJN+T5C1VVUlem+Q2Sb4/yT9K8pYkb6yqU7c8zYlJnp7kiUkemOTGSf7rlu/xg0l+NsnPJDktyV8meeq2UV6e5IeT3DDJ66vqgqr66e3BBwCwqQ4q1KrqplX141V1fpI/TXK3JD+R5Jbd/eTufkt3d5KHJLlPkkd397u6+4LufkaSjyR57Jan3JPkR9ePeW+Ss5M8pKr2zfMTSV7S3S/s7g9197OTvGvrTN19dXf/fnf/UJJbJvm59ff/8Hor3hOravtWuH0/z1lVdX5VnX9lLj+YVQAAcMwd7Ba1f53k+UkuT3KX7n5Ed/92d2+vnPsmuX6Sz653WV5aVZcmuUeSf7DlcZd3919u+fyiJMdntWUtSb4lydu3Pff2z/9ed1/S3b/R3Q9J8m1JbpHk15M8+gCPP6e793b33uNz4rX82AAAy9lzkI87J8mVSR6X5P1V9TtJXpbkj7r76i2Pu16STyf5J/t5ji9tuX3Vtvt6y9cfsqo6Mcn3ZbXV7uFJ3p/VVrlX7+T5AAAmOKgw6u6LuvvZ3f0Pk3xXkkuT/Lckn6yqX6iqf7R+6Luz2g15zXq359aPzxzCXB9M8oBty77m81r5x1X1wqwOZnhBkguS3Le7T+vu53f3Fw7hewIAjHLIW7C6+x3d/SNJTs1ql+hdk7yrqv5Jkjck+eMkr66q762qO1XVA6vqP6zvP1jPT3JmVT25qu5SVU9Pcv9tjzkjyf+f5JQkP5Tkdt39U93954f6MwEATHSwuz6/zvr9aeclOa+qbpHk6u7uqnp4Vkdsviir94p9Oqt4e+khPPcrq+qbkzw7q/e8/W6SX0zy+C0P+6Mkt+ruL339MwAAbL5aHax53XVK3aTvXw9degwAFrbndrddeoSN9Np3vmbpETbScade8CfdvfcbPc4lpAAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYSqgBAAwl1AAAhhJqAABDCTUAgKGEGgDAUEINAGAooQYAMJRQAwAYas/SAwDABFd94pNLj7CRHnbr+yw9woZszQaiAAACPUlEQVS64KAeZYsaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACGEmoAAEMJNQCAoYQaAMBQQg0AYCihBgAwlFADABhKqAEADCXUAACG2rP0AEuoqrOSnJUkJ+X6C08DALB/18ktat19Tnfv7e69x+fEpccBANiv62SoAQBsAqEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIYSagAAQwk1AIChhBoAwFBCDQBgKKEGADCUUAMAGEqoAQAMJdQAAIaq7l56hkVV1WeTXLj0HAdwsySfW3qIDWS9HTrrbGest52x3g6ddbYzk9fbHbr75t/oQdf5UJusqs7v7r1Lz7FprLdDZ53tjPW2M9bbobPOdmY3rDe7PgEAhhJqAABDCbXZzll6gA1lvR0662xnrLedsd4OnXW2Mxu/3rxHDQBgKFvUAACGEmoAAEMJNQCAoYQaAMBQQg0AYKj/DTxbtGy5NWJOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'esta es mi vida.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A3LLCx3ZE0Ls"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> ¿ todavia estan en casa ? <end>\n",
      "Predicted translation: are still still home ? <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAngAAAI5CAYAAAAyk0fzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcLXdd5//3J7tJCJCwy46i7NuVVRAmYnDjNwI/kDWAkiGAbAPMMIqACgwYdEBAyCD7IosgIGsQEHRABpA1bGExsiQQgYRAyPqZP+pc0+nbSW4g91b1t5/Px6MfOV3n9OlP1+Omz6urTlVVdwcAgHHsMfcAAABcvAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYATeAlTVz1bVe6rqBnPPAgBsfgJvGQ5PcvskD5x5DgBgANXdc8+wpVVVJflqkmOS/GaSK3X32bMOxaJV1RWS7LN2WXcfP9M4ACyQLXjzu0OSSyR5eJKzkvzavOOwRFV1yap6aVWdluTrSb6y7gMA/oPAm9/9kry+u3+Y5NWZdtfCekcluVGS/5zkR0nuleSxSb6W5B4zzgXAAtlFO6OqOiDJN5P8end/oKpunOSDmXbTfnfe6ViSqvpaknuu/p2ckuSm3X1cVd0zyQO7+44zjwjAgtiCN6+7Jjmpuz+QJN398SRfTPLbs07FEl0qyb+ubp+c5JDV7Q8mufUsEwFsAVV1QFXdr6ouOfcsF4XAm9d9k7xi3bJXxG5advSlJNdc3f5skt9eHaBzlyTfmW0qgPHdPcmLM71mbxp20c6kqq6S6c3x1+nuL65ZfuVMR9Vet7u/MNN4LExVPSrJ2d397Kr6T0n+Lsnemf5Ie0R3P2fWAQEGVVXvS3K5JD/s7m0zj7PTBB5sQlV11STbknyxuz819zwAI6qqqyf5QpKbJ/lQpvc/HzvnTDvLLtoZVdVVV7vZNrxvd8/D5tHdx3f3G8QdwC513yQfWL1H/m3ZRG+hsgVvRlV1dpIrdve31i0/JMm3unvPeSZjCarq0Ume190/Wt0+X939Z7tpLIAto6q+mOQp3f2SqrpLkmcnuUpvgngSeDOqqnOSXL67v71u+dWSHNvdB8wzGUtQVV9Jsq27/311+/x0d1/zAu4H4CKqqlsneVem1+kfVNU+SU5Ico/uPmbe6S7cXnMPsBVV1bNXNzvJ06rqh2vu3jPTvv6P7/bBWJTuvsZGtwHYLQ5P8qbu/kGSdPcZVfXaJPfPdHnRRRN487jB6r+V5DpJzlhz3xlJPpbpygWQJKmqG3X3J+aeA2ArqKp9M50e5Z7r7npFkndW1YHdferun2zn2UU7k9XBFa/NdBWC7889D8u22p3/mSQvT/Lq7v63mUcCGFZVXSbTteFfvv79dlV1nyTv7u4TZhluJwm8mVTVnpmuKXqjzXLINfOpqmsnuXemvyavmeQDmWLv9d19ypyzzamq9kvyiCSHZjpP1XnODNDdN5xjLoC5CbwZVdVxSe62OvwadkpV3SJT7N09yUFJ/q677z7vVPOoqhcl+a0kr0vyjUzva/0P3f3kOeYCmJvAm1FVHZ5pi8x9uvukuedhc1mF3vOT3HCrnlKnqr6T5O7d/e65ZwE2v9UZC3YqjJZ+9gIHWczrMUmukeTrVfW1JD9Ye6fdS6xXVddMcq9MW/B+JtOu2t+ddah5/TCJ9yMCF5e1l308MMmjk3w4yQdXy26V6UwXz9zNc11ktuDNqKqeeEH3273EdlX10ExRd4skn07yyiSv7O6vzzrYzKrq4Umul+TI7j5n7nmAcVTVS5J8obufum7545Ncr7vvM8tgO0ngwSZQVf+W5NWZjuhyebKVqnpLktsmOTnJsUnOXHt/d995jrmAza+qTsl07dnj1i3/mSQf6+6D5pls59hFC5vDVTfDpXFmcFKSN849BDCkHyS5fZLj1i2/faa3hyyawJvR6rInv5/pQIurJtl77f1b9Y3z7Gh73FXVlTL9W9ln3f3vn2OuuXX3A+aegc3B71t+DH+e5LlVtS3Jh1bLbpnpChdPmmuonSXw5vXHSe6R5GmZ/iE9NsnVk/x2kifMNxZLswq7V2faHdmZroKydoueFye4YH7fcpF09zOq6quZzrW5/VRUn01yeHe/drbBdpL34M1odTj2kd39jqr6fpIbd/eXqurIJId2991mHpGFWF3/8JAkD03yf5PcKcnlk/xRkkdthgtf7ypV9YCcu1Vm/ZbNRZ/GgN3H71u2mj0u/CHsQpfP9MbwJDk1yaVWt9+R5FdmmYil+qUk/627P5dpy923u/sNSf5bpi0TW1JVPTbT6Qo+mmlrzN9mOsr44CQvmm8yFsjvW35sVXWpqjp47cfcM10YgTev45NcaXX7uCSHrW7fKslps0zEUv1UpgMKkuQ7mS7LlUwvWFv5fIkPSnJEdz8+0xG0z1kdOfvMJFebdTKWxu9bLpKqulpVvb2qfpTk35N8e/Vx0uq/i+Y9ePN6Y6ZraH4oybOSvLqqHpTkp5P86ZyDsTifS/LzSb6a5ONJHrw6dcpDk2zlc+FdOdNJSJPpRXr7aQtevVr+oDmGYpH8vuWienGmLb0PzAaXQlw678FbkNWlp26T6cSKfzf3PCxHVd07yd7d/ZKqummm3UqHJDk90xt+XzfrgDOpqi9nup7zx6rq/yZ5UXf/ZVXdKdOJoA+ZeUQWqqpumeTW8fuW81FVpya5ZXd/eu5ZfhwCb0ZVdbsk/6e7z1q3fK8kt96qp77gwlXV/pm26B2/la9jXFUvTPK17n5SVT0409GRH0py0ySv7W5b8IAfS1V9Ksn9u/ujc8/y4xB4M6qqs5Ncsbu/tW75IUm+5bxMcMGqao8ke2z/I6mq7pHVVvAkL+juMy/o69k6quruSb7X3e9aff6HSY5I8plML+LfnHM+lqeq/lOS/57kIeuvZrEZCLwZVdU5SS7f3d9et/zaST6y9MugsGtV1U4fBdrdD9yVsyxVVV01yb+tv8pHVVWSq3T38fNMxtJU1bFJHtnd71q9zeH/JPnDTKccOqG77zXrgCzO6nQ6+2Y6z+jpSc6zt23pr9EOsphBVb15dbOTvKKqTl9z955Jrp/plw9b22XXfX67JOck2X4t2utnOhJ+K+/K/0qSKyb51rrlB6/usxWc7a6W5POr27+V5G9XJ7J9V5J3zjcWC/awuQf4SQi8efz76r+V5Ls57yH6ZyT5xyT/e3cPxbJ0929uv11Vj8/07+QB3f2D1bIDkvxVzg2+rWj9FT22OzDJj3bzLCzbj5JcYnX70Jx7nsST1yyH/9DdL517hp+EXbQzqqonJjlq+ws2nJ+q+mams+0fu2759ZL8fXdfYZ7J5lFVz17dfGimUxmsvfD3nklunuSM7r7N7p6NZaqqv810Psl/zHRpsqt39zeq6rAkz+7un5t1QBapqi6f5L5JrpXkCd19UlXdJsk3uvsr8053wZzoeF5/nDVb76rqClX1u1V16xlnYpkOzLknaV3rikn2382zLMENVh+V5DprPr9Bkp9J8rEk959rOBbpYZn2kNwtyYO7+xur5b8au2jZQFXdLNNu/Xsn+Z2ce57NOyZ5ylxz7Sxb8GZUVW9P8o7uflZVHZjpZLYHZHox/53uftmsA7IYVfWSTLuVHpvpNCBJcsskT0/y3u6+/zyTzauqXpzkEd19ytyzLM3qVDo3znTVk/P8Mb+6zB1wAarqvUne391PXB1wcaPu/nJV3SrJX3f3oq+WI/BmVFXfyrTb7VNVdb9Mh2PfKNNfC4/u7q18CSrWqKqfynT5rQcm2Xu1+KxM78F7THf/8Py+ditZrafbJPlid//r3PPMpap+OdPVPDY60XM7BRNcuKo6JcmNV1G3NvCunuRz3b3frANeCLto53WJJN9b3f6VJG9cnbfrPZn290OSpLtP6+6HZHrBvkmmE/ke3N0P2cpxV1UvqaqHrG7vk+nyZO9K8vmq+tVZh5vXs5K8NcmVu3uPdR9bMu6qap+qenJVfaGqflRVZ6/9mHs+Fum0JJfeYPnPZ8cj9xdH4M3r+CS3WR0NeViSY1bLD8553zQO252d6VQpZ61ub3WH5dxd1nfO9EfTFZI8afWxVV09yR+veZ8Z03ueD8+0JfycTG93eG6msxo8ZMa5WK43JXliVe27+rxXW++enuRv5hpqZwm8ef1Zkpcn+VqmC8ZvP5/Z7bK1T33BOlW1V1X9aabT6nwi07+P71bVM6pq7wv+6qFdOuf+JX2nJH+zujLMXye57mxTze+fkjgq9Lzunungihdk+uPoTd398CRPzPSmeVjvMZk2uHw708Fs/5jkuEyn1vmDGefaKc6DN6PufkFVfSTJVZMc093nrO76UqbD+GG7ZyS5Z5IHZ/olkyS3TfK0TH+oPWamueZ2QpLrr04jc1imS08l04FKW/kyZc9PclRVXSnTHwPnWRfd/bFZpprX5ZNsP83QqUkutbr9jkxbZOA8Vgdv/eLqkmU3zfS79mPd/e55J9s5Am8mVXXJJDfs7g8kWX8h4+/l3F9EkCT3SvLA7n7bmmVfqqpvJ3lhtm7gvSjJa5J8I9NWmb9fLb9FpqPSt6rXr/579Ab3dbbmFT6Oz3SqoeMzbYU5LNPv3lvlvCebh/O8Rnf3ezK9N377fbdJcmx3f3e2AXeCwJvPOUneXlWHdfc/bV9YVTfO9A/pp2ebjCW6ZKYtu+t9KeduidhyuvuPqurTmS5D9druPmN111nZ2ltlrjH3AAv0xkynGvpQpoNQXl1VD8r0u/ZP5xyMRdr0r9HegzeT7v5+pjdw3m/dXfdJ8s7uPmn3T8WCfSLJwzdY/ogkH9/NsyzNaUl+OckxVXWV1bJ9Mu2G25JWp4i5bqaDCN6e5JzVsjtmOhH0ltPdj+/up6xuvz7JLyb5iyR36e7fn3U4FmeE12iBN6+XJfn/t79Jvqr2yLQr7iVzDsUiPS7J4atTPLx0dXqQz2f6ZfPYmWebTVXdO8lrk3wh01ar7Qec7JFpnW1Ja9bLF3Pe9bJntuh6qaqnVNWDt3/e3f/c3X+W5MpV9cczjsZyberXaIE3r2MynQ5l+0XlD8205eEts020Caz+J9tqvprk2klel+kAgoNWt38u03uKtqrHJXlQdz8q027Z7T6U6SoOW5X1sqP7JvmXDZZ/NDtupdkyquo3qurI1TVXOa9N/Rq9FV8oF2N11Owrc+4vl/smec3qZMecjzVHG28lX0lyVnf/fnfftbvv0t1/kOT01X1b1c8m+eAGy0/NudeN3Iqslx1dLtPpLtb790xH2G45VfXfM7038Q+SfLKqbjDzSIuy2V+jBd78XpbkTqv3Dv1WkpfOPM/squq9VfXiqrr06vabq+rwueeaWWU6+nG9A5P8aDfPsiTfyLRlc73bZeODUrYK62VHx2c6tdB6t8t0LtKt6CGZrnv+05kOPDmmqn6lqq66OvfmFavqqjPPOLdN+xrtKNqZdfdnqupTSV6V5Gvd/eG5Z1qAT2c6v9mZq9uXSPLcqrrZ6sSkW0ZVPXt1s5M8rarWXuFkzyQ3z9Y+yOLoJM+uqt9dfX6VqrptpvMGPmm2qeZnvezoBUn+fHVJu+2nvDg007kkt+oR1wdndYL97n7q6u0vb1/d9wuZtl5dO1vztDpJNvdrtMBbhpcn+V9JHMmVpLt/b82nv5ckVfUXSd6xukzM67v7ZTOMNoftu0wqyXWSnLHmvjOSfCzJUbt7qKXo7meszld1TJL9krw3027ro7r7ubMONyPrZUfd/cyqukySZ2d6H1Uy/T/0rO5+xnyTzeoLmY62/mqSdPefVNXzMl3q7rOZdk3uP9dwC7IpX6Ore6O9PuxOVXVwppB5QXefMPc8S1VV107yvCTbuntLnfutql6c5BGrM6uzTlXtn+mFao9MJyDdsqdIWct62dHq2t/XzfRH05ZeJ1X1sCR36O67zj3Lkm3W12iBBwAwGAdZAAAMRuABAAxG4C1IVR0x9wxLZL3syDrZmPWyMetlY9bLjqyTjW3G9SLwlmXT/QPaTayXHVknG7NeNma9bMx62ZF1srFNt14EHgDAYLb8UbT71L69Xw6Ye4wkyZk5PXtn37nHWBzrZUfWycasl41ZLxuzXnZknWxsSevl+/nuSd192Qt73JY/0fF+OSC3qEPnHgMA4EK9u1//rzvzOLtoAQAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABrPpA6+q9p57BgCAJVlc4FXVnarqA1X13ar6TlW9s6qus7rv6lXVVXXPqnpPVZ2W5L+s7rt1Vf1DVf2wqr5eVX9ZVQfN+sMAAMxgcYGX5IAk/yvJzZPcPsnJSd5SVfuseczTkjwvyXWT/G1V3SDJu5K8OcmNktwlyY2TvGj3jQ0AsAx7zT3Aet39N2s/r6oHJDklU/B9bbX4L7r79Wse89Qkr+nuZ65ZdmSSf6mqy3X3t9Y95xFJjkiS/bL/Lvk5AADmsrgteFV1rap6VVV9qapOSXJipjmvuuZhH1n3ZTdLcp+qOnX7R5J/Wt13rfXfo7uP7u5t3b1t7+y7K34MAIDZLG4LXpK3JPl6pvfWfT3JWUmOTbJ2F+0P1n3NHklemOTPN3i+r++CGQEAFmtRgVdVhyS5TpKHdvd7V8tumguf82NJrtfdx+3iEQEAFm9pu2i/m+SkJA+qqp+pql9K8vxMW/EuyNOT3Lyqnl9VN1l97W9U1Qt29cAAAEuzqMDr7nOS3CPJDZN8OslzkzwhyekX8nWfTHK7JFdP8g9JPpHpSNsTd+G4AACLtKhdtEnS3e9Jcv11iw9cc7vO5+s+kuROu2ouAIDNYlFb8AAA+MkJPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDB7DX3AHOrPffMngddcu4xFqcOPHDuERbnyw+82twjsIlc469PnHsENomzj/vq3CMsU58z9wTL1Dv3MFvwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABrNpA6+q3ldVz9nZzwEAtoq95h7gwlTV/ZM8p7sPXHfXXZKcufsnAgBYtsUH3vnp7u/MPQMAwBItZhdtVd2uqj5UVadW1clV9c9V9bAkL05yQFX16uNJq8fbBQsAsIFFbMGrqr2SvCnJXyW5d5K9k9w0yWeSPDLJU5Nca/XwU+eYEQBgs1hE4CU5KMmlkrylu7+0Wva5JKmqmyTp7j7h4vpmVXVEkiOSZL89Dri4nhYAYBEWsYt29X66lyR5Z1W9taoeXVVX2YXf7+ju3tbd2/apn9pV3wYAYBaLCLwk6e4HJLlFkvcnuXOSL1TVYfNOBQCw+Swm8JKkuz/R3U/v7tsneV+Sw5OckWTPOecCANhMFhF4VXWNqvqfVXXrqrpaVd0hyQ2THJvkq0n2q6o7VtVlqmr/WYcFAFi4pRxk8cMk107yuiSXSXJiklcmeXp3n1lVz0/y6iSHJHlykifNNCcAwOItIvC6+8RMV6Y4v/uPTHLkumW3vyifAwBsFYvYRQsAwMVH4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMRuABAAxG4AEADEbgAQAMZq+5B5hbn312zj75lLnHWJ7vnTz3BItz1T85Ye4RFunb/+Xmc4+wSJ993KXnHmGRrvy2PeceYXEOOuXUuUdYpLNOOHHuETY1W/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABrNpA6+q3ldVz9nZzwEAtoq95h7gwlTV/ZM8p7sPXHfXXZKcufsnAgBYtsUH3vnp7u/MPQMAwBItZhdtVd2uqj5UVadW1clV9c9V9bAkL05yQFX16uNJq8fbBQsAsIFFbMGrqr2SvCnJXyW5d5K9k9w0yWeSPDLJU5Nca/XwU+eYEQBgs1hE4CU5KMmlkrylu7+0Wva5JKmqmyTp7j7h4vpmVXVEkiOSZL/sf3E9LQDAIixiF+3q/XQvSfLOqnprVT26qq6yC7/f0d29rbu37Z19d9W3AQCYxSICL0m6+wFJbpHk/UnunOQLVXXYvFMBAGw+iwm8JOnuT3T307v79knel+TwJGck2XPOuQAANpNFBF5VXaOq/mdV3bqqrlZVd0hywyTHJvlqkv2q6o5VdZmq8qY5AIALsJSDLH6Y5NpJXpfkMklOTPLKJE/v7jOr6vlJXp3kkCRPTvKkmeYEAFi8RQRed5+Y6coU53f/kUmOXLfs9hflcwCArWIRu2gBALj4CDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwew19wCL0D33BGwG55w99wSLdNm//ODcIyzSJQ/bNvcIi3TSDface4TF+eZdrzT3CIt07SNPm3uEZTp55x5mCx4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgLtbAq6r3VdVzLs7nBADgorEFDwBgMAIPAGAwuyLw9qiqp1bVSVX1rao6qqr2SJKqunRVvbSqvltVp1XVu6vqetu/sKruX1WnVtWvVtXnquqHVfXmqrpkVd2tqr5YVSdX1cur6qfWfF1V1eOq6kur5/1UVd1nF/xsAACLtysC795Jzkpy6yQPS/LIJPdY3feSJLdI8v8luXmSHyZ5x9pYS7Jvkv+6ep5Dk2xL8vokhye5a5L/nOQ3kjxkzdf8SZLfSfLQJNdN8rQkL6iqX7/YfzoAgIXbaxc857Hd/Yer21+oqgclObSqPpLkzkl+qbvfnyRVdd8kx2eKuReumemh3f351WNeleRRSS7f3Setlr0pyR2SPLOqDkjy6CS/0t0fWD3HV6rq5pmC763rB6yqI5IckST7Zf+L9YcHAJjbrgi8T677/BtJLpfkOknOSfLB7Xd098lV9alMW922O3173K2cmOSE7XG3Ztn2r7lukv0ybQnsNY/ZO8lXNxqwu49OcnSSHFQH90aPAQDYrHZF4J257vPOtCu4LuBr1kbWWRvcd37PmTX//c1MWwMvaBYAgOHtisA7P8dmirFbJdm+i/agJDdI8uKf8HlPT3K17n7PTzokAMBmt9sCr7u/uHrv3AtW74H7XpKnJDklyat+guf9flUdleSoqqpM8XhgklsmOWe1OxYAYMvY3efBe0CSDyd58+q/+ye5U3ef9hM+7xOSPCnJY5J8JskxmY64/cpP+LwAAJtOdW/tYwwOqoP7FnXo3GMAgznjsG1zj7BIJ91gn7lHWJzTbvKTbuMY07WP/NLcIyzSO09+0Ue7+0J/wbiSBQDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGD2mnsAgBHt94Fj5x5hka7y4X3nHmFxLvO2s+ceYZE+/qDrzz3CMh21cw+zBQ8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDBDBV5VPayq/qWqflBV/1ZVj597JgCA3W2vuQe4mB2a5A+TfCbJ7ZK8sKo+091vnncsAIDdZ6jA6+7fWvPpl6vqqUmuMtc8AABzGGoX7VpV9T+S7J3kDXPPAgCwOw21BW+7qvqDJA9Pcsfu/uYG9x+R5Igk2S/77+bpAAB2reECr6oOSfJHSX69uz++0WO6++gkRyfJQXVw78bxAAB2uRF30V49SSX57MxzAADMYsTA+2ySX0jyjbkHAQCYw4iBd/0kr0hy2bkHAQCYw4iBt3+Sn8t0BC0AwJYz3EEW3f2+TO/BAwDYkkbcggcAsKUJPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPAB9jSkoAAAHFElEQVSAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDBCDwAgMEIPACAwQg8AIDB7DX3AAAjOue00+YeYZl+dPrcEyzOiY+63twjLNIn3/C8uUdYpD2P2rnH2YIHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwGIEHADAYgQcAMBiBBwAwmE0TeFX1mKr66txzAAAs3aYJPAAAds7FEnhVdVBVXerieK6L8D0vW1X77c7vCQCwGfzYgVdVe1bVYVX1qiQnJLnRavklq+roqvpWVX2/qv6hqrat+br7V9WpVXVoVX26qn5QVe+tqmuse/7HVdUJq8e+LMmB60b4tSQnrL7XbX7cnwMAYDQXOfCq6npV9Ywkxyd5TZIfJLlTkvdXVSV5a5KfTvIbSW6S5P1J3lNVV1zzNPsmeXySBya5VZJLJXn+mu9x9yR/kuSJSW6a5PNJHr1ulFckuVeSSyQ5pqqOq6o/XB+KAABbzU4FXlUdUlUPr6qPJPmXJD+f5JFJLt/dD+ru93d3J7lDkhsnuVt3f7i7j+vuJyT5cpL7rnnKvZI8dPWYTyY5Kskdqmr7PI9M8tLufkF3f6G7n5Lkw2tn6u6zu/tt3X3PJJdP8tTV9//iaqvhA6tq/Va/7T/PEVX1kar6yJk5fWdWAQDAprGzW/B+L8mzkpye5Ge7+87d/bruXl9HN0uyf5Jvr3atnlpVpya5fpJrrXnc6d39+TWffyPJ3pm25CXJdZJ8cN1zr//8P3T397v7Rd19hyS/kORySf4qyd3O5/FHd/e27t62d/a9gB8bAGDz2WsnH3d0kjOT3C/JZ6rqjUlenuTvu/vsNY/bI8mJSW67wXOcsub2Wevu6zVff5FV1b5Jfj3TVsJfS/KZTFsB3/TjPB8AwGa2U0HV3d/o7qd0988l+eUkpyb56yRfq6pnVtVNVg/9WKbdpeesds+u/fjWRZjrs0luuW7ZeT6vyS9W1QsyHeTxnCTHJblZd9+0u5/V3d+9CN8TAGAIF3mLWXd/qLuPTHLFTLtur53kw1V12yTvTvJPSd5UVb9aVdeoqltV1ZNX9++sZyU5vKoeVFU/W1WPT3KLdY+5T5J3JTkoyT2TXKW7H9vdn76oPxMAwEh2dhftDlbvv3t9ktdX1eWSnN3dXVW/lukI2P+d6b1wJ2aKvpddhOd+TVVdM8lTMr2n781J/izJ/dc87O+TXKG7T9nxGQAAtq6aDn7dug6qg/sWdejcYwCjqZp7gmUqF1Dawc2vN/cEi/TON+z0dqEtZc8rHvfR7t52YY/zfxoAwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGAEHgDAYAQeAMBg9pp7AIAhdc89wTL12XNPsDwf+uTcEyzSYVe68dwjLNRxO/UoW/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAYj8AAABiPwAAAGI/AAAAaz19wDzKGqjkhyRJLsl/1nngYA4OK1JbfgdffR3b2tu7ftnX3nHgcA4GK1JQMPAGBkAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMAIPAGAwAg8AYDACDwBgMNXdc88wq6r6dpJ/nXuOlcskOWnuIRbIetmRdbIx62Vj1svGrJcdWScbW9J6uVp3X/bCHrTlA29Jquoj3b1t7jmWxnrZkXWyMetlY9bLxqyXHVknG9uM68UuWgCAwQg8AIDBCLxlOXruARbKetmRdbIx62Vj1svGrJcdWScb23TrxXvwAAAGYwseAMBgBB4AwGAEHgDAYAQeAMBgBB4AwGD+H5cUSqupxsgaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'¿todavia estan en casa?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DUQVLVqUE1YW"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> trata de averiguarlo . <end>\n",
      "Predicted translation: try to be to sing . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAKICAYAAACsbvkCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmUZQV5rvHnhW4giENExcaICMgQR7QFiYriGCRmUK4rJDEiiWiQiItoEjW5agwighoMTjhAcDZGF05xhiARIeAQEZRB0auAgCIySDfCd//Yp6W67KEauuvbp+v5rVWrq3adOv31WXCe2nOqCkmS1GOT7gEkSVrIDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0WdQ8gSXOVZAtgJ6CAi6vqxuaRpNvNNWJJo5dkUZKjgauBbwDfBK5O8toki3unk24f14glTYPXAgcAzwNOnyx7NHAkwwrFi5rmkm63eK1pSWOX5HLgoKr61Kzl+wHvqKolPZNJt5+bpiVNgzsDF69i+cXAXeZ5Fmm9MsSSpsE3gBesYvlhwNfneRZpvXLTtKTRS7I38CngUuAMhqOm9wK2BfatqtPX8OPSqBliSVMhybbA84FdgQDnAW+uqktbB5NuJ0MsSVIjT1+SNEpJHjrXx1bVVzfkLNKG5BqxpFFKcgvDvuCs5aFVVZvOw0jSBuEasaSxum/3ANJ8cI1Y0qhNLmF5BPCmqvp+9zzS+maIJY1ekuuAB1TVJd2zSOubF/SQNA0+AzyuewhpQ3AfsaRp8AXg1UkeBJwDXD/zm1X1kZappPXATdOSRm9yBPXqeNS0ppohliSpkfuIJUlq5D5iSVMhyV2B3wW2Azab+b2q+qeWoaT1wE3TkkYvySOATwLLgLsDPwKWTL6+pKoe1DiedLu4aVrSNDgaeC9wL+BGhlOZtgPOBo5qnEu63VwjljR6Sa4BHl5VFyT5GbBXVZ2f5OHA+6rqfs0jSreZa8SSpsHyGZ//GLjP5PPrgG3nfxxp/fFgLUnT4KvAw4ELgFOBf06yDfBnwP82ziXdbm6aljR6SZYCd6yqU5LcHTgJeCRDmJ9dVd9sHVC6HQzxCCS5H/A24DDfUCRpYXEf8Tg8C3gscFDzHJKkeeYacbMkAS4BPgc8Fdi2qm5uHUoamSTfBFb7ZuV5xJpmHqzVbx/gjsALgH2BpwAfb51IGp8Pz/p6MfAQhv3Eb5r/caT1xzXiZklOBJZX1cFJjgG2r6r9m8eSpkKSFwP3qapDu2eRbitD3CjJHYDLgP2q6ktJHgKcwbB5+ure6aTxS7IjcHZV/Wb3LNJt5cFavZ4OXFVVXwKoqq8DFwJ/3DqVND32Bm7oHkLjkOQOSf48yZ27Z1kX7iPu9UzgPbOWvYfhKOq3zP840jgl+djsRQw3fdgdeOX8T6SRegbwDuAw4LjmWebMTdNNktwb+B6wW1VdOGP5bzEcRf3bVXVB03jSqCQ5YdaiW4ArgS9W1WcbRtIIJTkVuAdwQ1UtbR5nzgyxJGnqJdme4UprewBfAR5aVed1zjRX7iNulGS7yXnEq/zefM8jSVPsmcCXJsfafIphF99UcI24UZKbgSVVdcWs5VsDV1TVpj2TSeOS5Hus+oIexXB/4ouAd1bV7H3JWiCSXAgcUVUnJnka8Ebg3jUFkXONuFdY9ZvLVgxvLpIGJwB3ZTir4D2Tjwsnyz4G3Ax8JIlnHCxASX6H4eC9f58s+gSwJfCEtqHWgUdNN0jyxsmnBRyZZObpF5sy7OP4+rwPJo3XDsBrquo1Mxcm+VuGAxufluSlwN8BH+gYUK2eBZxcVdcDVNXyJB8CDmS4fPCouWm6QZJTJp8+huECHjNver6c4ajpY2YeTS0tZEl+znDwzUWzlu8EfLWq7pRkF+CcqtqqZUi1SLI5cDlwQFV9esbyRwGfAbapquu65psL14gbVNU+k4O0PgQcVFXXds8kjdwNwKMZ9gXP9GhuvaDHpsAv5nMojcIdGc4b/szMhVV1epLnMuzqG3WIXSNukmRThv3AD56WQ+ylLkleAvxf4F3A/zDs1tmDYdPjq6rqNUkOB/atqie2DSrdBoa4UZKLgP0nh9tLWoPJgVgvAHadLPo2cGxVfXDy/d8Aqqo80FFTxRA3SvIs4ADgz6rqqu55JGlarOGUtl9TVTts4HFuF/cR93oRcF/gR0l+CFw/85ve7FySVmvmtaS3Ag4HzmI4ABZgL4bdF6+b57nWmSHuNftm55ImJkdK71BVVyW5ljWs/VTVneZvMo1BVf0qsJP7uh9VVa+e+ZjJsQX3n+fR1pmbpjUKSfZh2Ey/HbDZzO9V1eNahlKrya6bD1TVssnnq1VV/zZPY2mE5nJ6W89kc+MasdolORB4K/BR4LHAycDODJvtZ98mUgvEirgmWcRwp6Uzq+onvVNppK5neO+YfXrbY5mC+1Ub4kZJNgNexq1rgotnfn8BXWv6RcChVfWOySbIl1TVd5Mcx8jP/9OGV1W/TPIRhqOlDbFW5Q3Am5IsZbjzEsAjGK649YquoebKa033ehXDfyivY7i/6ouBNzG82RzSONd82wH4/OTzZQwHXsBwMMaBHQNpdL4B7NQ9hMapql7LcPelBwKvn3w8EHhWVR3VOdtcuEbc6xnA86rq00mOYbhW6sVJzgeeCLytd7x58xOGq+MA/Ah4APC/wNbAb3QNpVF5BfC6JC8HzuHXzzD4acdQGo+q+hDD1QqnjiHutQ2w4qpa1wF3mXz+aWD0v8WtR18CngR8k+F/pDcmeSLweKbggu2aF5+c/PkRVj56esUdzBbKbhytRZK7MGtr79h/UTPEvX4AbDv58yLgyQy/7e/Fwrpm7qHAFpPPjwR+CTySIcr/3DWURmWf7gE0Xknuw3DA5z6sfKzNVPyi5ulLjZIcCVxXVUck2R94P/BD4F7A0VX1stYBJWkKJPkiwxbFY4BLmXXOeVX9V8dcc2WIRyTJngxrghdU1Se655kvSW4GllTVFbOWbw1csYCOHtcaJHkg8FxgR4a7ll2W5A+B71fV13qnU6ck1wGPqKpzu2e5LTxqulGSvSfnSAJQVWdW1euBTyfZu3G0+ZbVLN+cle/VrAUqyZMY7rp0L+Bx3HoQ347Ay7vm0mh8j+H9Yiq5j7jXKcAS4IpZy+88+d5GvSY4uW0dDJuRnjf5rXaFTRnuNfvteR9MY/Qq4PCqevPkXPMVTgX+pmckjchhwJFJDpl9da1pYIh7rTiQYLatmXV6xkbqryd/BvhL4OYZ31sOXAI8b55n0jjdH/jUKpb/FLjrPM+i8TmZYY34O0mWMRzw+Ste4lK/JsnHJp8W8J7JfzgrbMpwHu2X532weVZV9wVIcgrwtKq6unkkjdfVDJulL5m1/KEMBzhqYTu0e4DbwxD3WHGZvjC8wcw8VWk5cDrw9vkeqktVeWqK1uZ9wNFJnsHwC+yiJI9hOEr2hNbJ1G7ab/rhUdONJlcJOqaqFsJm6DVKsjOwP6u++9JBLUNpNJIsBk4E/pjhF9hbJn++Dziwqm5e/U9rIUiyDcNlLncE/nFy+8xHApdW1fd6p1szQ9woySYAVXXL5Ot7Ar8HnFdVG/2m6RWS7Af8B/A14GEMR8fuyLDP50tV9fuN42lEkuwI7M5wxsfXqurC5pE0AkkeBnyB4ejp+wO7Tm4c8wpg56r6k8751sbTl3p9kskBS0m2As4Gjgb+K8mfdw42z/4JeGVV7cVw04dnAtsz3Aji1L6xeiV5YJLjkvxnkiWTZX+YZPfu2eZbkj9IsqiqLq6qD1fVh4ywZjgGOLaqdmd4D1nhMwzXZhg1Q9zrYcAXJ58/Dfg5cA/gOQy3BlwodgE+OPn8JmDLqrqRIdAvbJuqkefN/pr3A5cneUuS3+keRqPzMGBV+4kvY7im/6gZ4l53BH42+fxJwEer6iaGOO/YNtX8u5ZbrzV9Gbfe7m4R8JstE/Vbcd7sH7HyRU1OBfZomajXNgy3Cd0JOC3Jd5O8KskuzXNpHH7Bqt8rduXXr9MwOoa41w+ARya5A8MNH1bcaeiuwA1tU82/M4FHTT7/JLfe7u4E4Iy2qXp53uwMVXVtVZ1QVU8E7s1wr+p9gfOSnNU7nUbgZODlSVZcXauSbM9wF7v/6Bpqrgxxr9cD72Y4D/JHwGmT5Xsz3BJwoTgc+Mrk81cAnwWeznBHqr9smqnbivNmZ1vw581W1WUMIT6S4b7VD+udSCPwIoZfUK8EtmQ4BfQi4BrgHxrnmhOPmm42OdpvO+BzVXXdZNl+wM+q6r9bh5sHk2ttPwk4s6p+srbHLxRJjmK4xOczGO5ZvZThcqgnAidU1T/1TdcnyT7AnzL8ogbwUeDdVXVK31QaiySPY/hldRPgq1X1+eaR5sQQN0lyZ+BBVfWlVXzvkQynMC2IK00luZHhdINLumcZi9WcN7sJ8F4W4HmzSY5meC3uwXAk7HuAk6tq2Rp/UBu9jeG91BA3SXJHhgOTnjxzzTfJQxj2md6rqq7qmm8+JTkTeNm0/PY6n5LswK2/4S/Y82aTfJkhvh+oqp92z6Px2BjeSw1xoyTvBa6rqufOWHYMwwnoC+YiFkn2BV7DcFrOOcy64cVCeeNN8q65PnYhXm1sshtjD1Z99bWTWobSKEz7e6khbpTkyQznR25TVTdNrrT1Q+DQqvpI73TzJ8ktM76c+R9kgKqqjfp2kCsk+fisRXszbJJeceDeAxjWjE+bhjeX9WlymtLHgR0Y/ru4meH0tpuAZWO/u442rGl/L/WmD70+x3Ca0lOBjwCPZ/hNf/Yb8sbu2cD/Y+XbIMIQne3mf5weVfXUFZ8neQnDuZHPXnEt8slpbu9kYR1Rv8KxwFcZLm95OfAQhvt2v4UpOCpWG9xUv5e6RtxscnTsLlX1h0lOAq6tqud3zzWfktwMLKmqK2Yt3xq4YqGsEc+U5DLg8VV13qzl9we+UFX37JmsR5KfAI+pqnOTXAPsUVXfmdyB6V+r6kHNI6rZNL+Xukbc7yTgnCT3Bv6I4Te5hSasvEl6ha2AG+d5lrHYCtiW4dSlmZYwnCe50IRbL3JzJcM51t9h2Py40+p+SAvK1L6XGuJmVfWtJN9kuJ3bD6tqwVwlKMkbJ58WcGSSmVcT25ThwJyvz/tg4/AfwAlJXsytFzt5BMOVgka/z2sDOBd4MPBd4Czg7yZbUp7DcOEGLXDT/F5qiMfh3cC/AC/rHmSePXDyZ4DdWPmayssZ9gkeM99DjcRfAa9jOJd48WTZLxn2ES+kG4KscARwh8nn/wB8AjgFuIrhoicCkpwP3K+qFup7+1S+l7qPeASS3JXhdohvq6rLu+eZb0lOAA6rqp93zzI2kwO0dmT4ZeWiFQdu6Vf/31xdvon9SpJDga2r6pXds3SY1vdSQyxJUiNv+iBJUiNDLElSI0M8EkkO7p5hTHw9VubrsTJfj5X5eqxs2l4PQzweU/Ufzjzw9ViZr8fKfD1W5uuxsql6PQyxJEmNFvxR05tl89riV6cn9rmJZSxm8+4xRsPXY2W+Hivz9ViZr8fKxvJ6XMvVV1XV3df2uIV60vevbMEd2DNTcyU0SdKU+Hx9+PtzeZybpiVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJamSIJUlqZIglSWpkiCVJajTqECc5Nclx3XNIkrShjDrEc5FkcfcMkiTdVqMNcZITgccAz09Sk48DJ38+JclZSZYDz01yc5Kls37+OUmuSrJZx/ySJM3Fou4B1uAwYGfg28BLJ8vuP/nzKOBvgIuAa4GnAgcBZ8/4+YOAd1fV8nmZVpKk22C0a8RVdQ2wHLihqi6vqsuBmyfffkVVfbaqvltVVwJvBw5IsgVAkt2ARwDvXNVzJzk4ydlJzr6JZRv+HyNJ0mqMNsRrcfasr09miPbTJl8fBJxVVeeu6oer6viqWlpVSxez+QYcU5KkNZvWEF8/84uqugk4CTgoySLgmaxmbViSpDEZ8z5iGNZyN53jY98OnA8cAtwR+MCGGkqSpPVl7CG+BNgjyfbAdaxhDb6qLkhyOnA08IGq+vl8DChJ0u0x9k3TxzCsFZ8HXAlst5bHvxPYDDdLS5KmxKjXiKvqAmCvWYtPXMOPLAEurKrTNthQkiStR6MO8Vwl2QrYleHc4yOax5Ekac7Gvml6ro4D/nvy8bbmWSRJmrONYo24qg4EDmweQ5KkdbaxrBFLkjSVDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0WdQ/QLQmbbLFF9xijccnfP7R7hFHZ5qybukcYlWt2WNw9wqgsec+3ukcYlZuv+Xn3CONSc3uYa8SSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1MsSSJDUyxJIkNTLEkiQ1mroQJzk1yXHdc0iStD5MXYglSdqYTFWIk5wIPAZ4fpKafGyfZO8kZya5McmPk7whyWbN40qStFZTFWLgMOAM4ARgyeTjJuA/ga8BuwN/ARwAHNk0oyRJczZVIa6qa4DlwA1VdXlVXQ4cAlwGHFJV51fVJ4C/Bw5NsuWqnifJwUnOTnL2cpbN2/ySJM02VSFejd2AM6rqlhnLTgc2A3Za1Q9U1fFVtbSqlm7G5vMxoyRJq7QxhDhAreZ7q1suSdIoTGOIlwObzvj6PGCvJDP/LY+aPO7i+RxMkqR1NY0hvgTYY3K09N2ANwPbAm9OsluS/YDXAMdV1Q2Nc0qStFbTGOJjGNZ2zwOuBBYD+zIcMf114F3A+4GXdg0oSdJcLeoeYF1V1QXAXrMWXwLsOf/TSJJ0+0zjGrEkSRsNQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUiNDLElSI0MsSVIjQyxJUqNF3QN0qypuufHG7jFGY7tXfLl7hFHZ9G5bd48wKj/cZ+fuEUblZ6/epXuEUdn1Red2jzAu18/tYa4RS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktRoKkKc5NQkx3XPIUnS+jYVIZYkaWNliCVJajRNIV6U5NgkV08+jk6yCUCSzZIcleSHSa5P8j9Jntw9sCRJazNNIf5Thnn3Ap4LHAy8cPK9E4DHAH8CPBD4N+DjSR7cMKckSXO2qHuAdXAZ8IKqKuDbSXYGDk9yMnAAsH1V/WDy2OOSPIEh2IfMfqIkBzOEnC3Ycl6GlyRpVaZpjfgrkwivcAZwL+BRQIDzkly34gPYD9hxVU9UVcdX1dKqWrqYzTf44JIkrc40rRGvSQEPB26atfwXDbNIkjRn0xTiPZNkxlrxI4BLGdaMA9yzqk5pm06SpNtgmjZNbwv8S5JdkuwPvBh4Q1VdALwXODHJ/kl2SLI0yYuSPK11YkmS1mKa1ojfC2wKnMmwKfqdwBsm33s28DLgtcBvAT8FzgJcQ5YkjdpUhLiqHjvjy0NX8f2bgFdMPiRJmhrTtGlakqSNjiGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKnRou4BpDG7+aqfdI8wKju99JzuEUbloG99p3uEUXnro/5P9wjj8pm5Pcw1YkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhpNXYiTnJrkuO45JElaH6YuxJIkbUymKsRJTgQeAzw/SU0+tk+yd5Izk9yY5MdJ3pBks+ZxJUlaq6kKMXAYcAZwArBk8nET8J/A14Ddgb8ADgCObJpRkqQ5m6oQV9U1wHLghqq6vKouBw4BLgMOqarzq+oTwN8DhybZclXPk+TgJGcnOfsmls3b/JIkzTZVIV6N3YAzquqWGctOBzYDdlrVD1TV8VW1tKqWLmbz+ZhRkqRV2hhCHKBW873VLZckaRSmMcTLgU1nfH0esFeSmf+WR00ed/F8DiZJ0rqaxhBfAuwxOVr6bsCbgW2BNyfZLcl+wGuA46rqhsY5JUlaq2kM8TEMa7vnAVcCi4F9GY6Y/jrwLuD9wEu7BpQkaa4WdQ+wrqrqAmCvWYsvAfac/2kkSbp9pnGNWJKkjYYhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSp0aLuASRNj7ppefcIo/Ku375f9wij8sUfvKN7hFHZdMncHucasSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjUYX4iSPTVJJ7tY9iyRJG9roQgx8GVgC/KR7EEmSNrRF3QPMVlXLgcu755AkaT60rREn2TvJV5Jcl+SaJGcmecDsTdNJDpw85vFJzk1yfZJTktx31vO9JMmPJ489KcnLk1zS8o+TJGmOWkKcZBFwMnA68GBgT+BY4ObV/MjmwEuAg4C9gLsAb53xfH8MvBx4GfBQ4Hzg8A00viRJ603Xpuk7McT041V18WTZtwGSbLOKxy8Cnl9V35k85hjghCSbVNUtwGHAiVX1jsnjj0yyD7Dzqv7yJAcDBwNswZbr6Z8kSdK6a1kjrqqfAicCn0nyySSHJ7n3Gn5k2YoIT1wKLGaIOcCuwFmzfubMNfz9x1fV0qpaupjN1/0fIEnSetK2j7iqns2wSfo04PeBC5I8eTUP/+XsH5/8uckqlkmSNDVaT1+qqm9U1VFV9VjgVOBZt/Gpvg3sMWvZ7K8lSRqdln3EkyOenwt8DPgRsAPwIOAtt/Epj2XYZ/w/wJeAP2JY27769k8rSdKG03Ww1g0MB1L9O3A34MfAe4GjgEeu65NV1QeS7AC8BtgS+AjDUdV/sL4GliRpQ0jVxrlrNclHgUVV9dQ1Pe5OuWvtmcfP01SSNiZZNLprIrX69A/O7h5hVDZdctE5VbV0bY/bKP4rSrIl8FfApxkO7Ho6w9rw0zvnkiRpbTaKEDMcMb0v8FLgN4ALgWdW1Udbp5IkaS02ihBX1S+AJ3TPIUnSuhrj3ZckSVowDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjRZ1DyBJ0yqLfAud6fcu2Ld7hJH51zk9yjViSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGi3qHqBDkoOBgwG2YMvmaSRJC9mCXCOuquOramlVLV3M5t3jSJIWsAUZYkmSxsIQS5LUaKMNcZJDk3y7ew5JktZkow0xcDdgl+4hJElak402xFX1iqpK9xySJK3JRhtiSZKmgSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqZEhliSpkSGWJKmRIZYkqdGi7gEkaVrdsmxZ9wijsskB1T3CVHKNWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRoZYkqRGhliSpEaGWJKkRlMT4iQvSnJJ9xySJK1PUxNiSZI2RuslxEnulOQu6+O51uHvvHuSLebz75QkaX27zSFOsmmSJyd5H3A58ODJ8js/FXnQAAAFUUlEQVQnOT7JFUmuTfJfSZbO+LkDk1yX5PFJzk1yfZJTktx31vP/bZLLJ489Cdhq1ghPAS6f/F2PvK3/DkmSOq1ziJPcP8lrgR8AHwSuB34XOC1JgE8C9wJ+D9gdOA34YpIlM55mc+AlwEHAXsBdgLfO+DueAfwz8HLgocB3gMNnjfIe4E+AOwKfS3JRkv87O+iSJI3ZnEKcZOskL0hyNvA1YFfghcA2VfWcqjqtqgrYB3gIsH9VnVVVF1XVPwLfBZ454ykXAc+fPOZ/gWOAfZKsmOeFwL9V1duq6oKqOgI4a+ZMVXVzVX2qqg4AtgFePfn7L5yshR+UZPZa9Ip/z8FJzk5y9k0sm8tLIEnSBjHXNeK/Bo4FlgH3q6rfr6p/r6rZFXsYsCVw5WST8nVJrgMeAOw443HLquo7M76+FFjMsGYMsBtwxqznnv31r1TVtVX1rqraB3g4cA/gncD+q3n88VW1tKqWLmbzNfyzJUnasBbN8XHHAzcBfw58K8lHgXcDX6iqm2c8bhPgx8CjV/EcP5/x+S9nfa9m/Pw6S7I5sB/DWvdTgG8xrFWffFueT5Kk+TKn8FXVpVV1RFXtAjwBuA74APDDJK9LsvvkoV9l2Ex8y2Sz9MyPK9ZhrvOBR8xattLXGTwqydsYDhY7DrgIeFhVPbSqjq2qq9fh75Qkad6t8xpoVX2lqv4KWMKwyXpn4KwkjwY+D/w3cHKSfZPcN8leSV45+f5cHQs8K8lzktwvyUuAPWc95s+AzwJ3Ag4A7l1VL66qc9f13yRJUpe5bpr+NZP9wx8GPpzkHsDNVVVJnsJwxPPbGfbV/pghzietw3N/MMkOwBEM+5w/BrweOHDGw74A3LOqfv7rzyBJ0nTIcLDzwnWn3LX2zOO7x5A0jZLuCUZl0T236R5hVD596XHnVNXStT3OS1xKktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUaFH3AJI0taq6JxiVX152efcIU8k1YkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhoZYkmSGhliSZIaGWJJkhot6h6gQ5KDgYMBtmDL5mkkSQvZglwjrqrjq2ppVS1dzObd40iSFrAFGWJJksbCEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1MgQS5LUyBBLktTIEEuS1ChV1T1DqyRXAt/vngO4G3BV9xAj4uuxMl+Plfl6rMzXY2VjeT3uU1V3X9uDFnyIxyLJ2VW1tHuOsfD1WJmvx8p8PVbm67GyaXs93DQtSVIjQyxJUiNDPB7Hdw8wMr4eK/P1WJmvx8p8PVY2Va+H+4glSWrkGrEkSY0MsSRJjQyxJEmNDLEkSY0MsSRJjf4/AqvDktW0fvoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 错误的翻译\n",
    "translate(u'trata de averiguarlo.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RTe5P5ioMJwN"
   },
   "source": [
    "## 下一步\n",
    "\n",
    "* [下载一个不同的数据集](http://www.manythings.org/anki/)实验翻译，例如英语到德语或者英语到法语。\n",
    "* 实验在更大的数据集上训练，或者增加训练周期。"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "nmt_with_attention.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
